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TITLE OF THE INVENTION 
ULTRASONIC PICTURE PROCESSING METHOD AND ULTRASONIC 
PICTURE PROCESSING APPARATUS 

BACKGROUND OF THE INVENTION 

The present invention relates to an ultrasonic 
picture processing method and an ultrasonic picture 
processing apparatus for determining and collecting 
pictures which are important to the diagnosis of a 
disease and which are not affected by the movement of a 
subject's body and by the movement of inspector's hands 
from ultrasonic pictures obtained by an ultrasonic 
diagnosis apparatus, and to an ultrasonic picture 
processing apparatus. 

The present invention also relates to an picture 
processing apparatus for extracting the contour of a 
target and displaying a picture capable of determining 
whether or not the extracted picture is fitted to the 
target . 

In recent years, as people's eating habits are 
improving, the number of people suffering from diseases 
such as obesity and hypertension is increasing in Japan 
Since the mortality rate of cardiac diseases resulting 
from these diseases is a second highest next to cancer, 
they are becoming more and more serious problems. In 
the diagnosis of a cardiac disease, an electrocardio- 
gram is used as a primary diagnosis method. For 
further detailed picture-using diagnosis, diagnosis 
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using an ultrasonic diagnosis apparatus is widely 
applied due to convenience and low cost (compared to 
other diagnosis apparatuses such as X-ray CT, MRI and 
PET) . 

5 When diagnosing of a cardiac disease using the 

ultrasonic diagnosis apparatus, real-time moving 
pictures obtained by the apparatus are used in most 
cases. A doctor observes the movement of the cardiac 

S4 wall of a patient for one piece of diagnostic informa- 

\l 

fQ 10 tion and diagnoses whether the patient's heart is 

abnormal. In case of a cardiac disease such as cardiac 

"sHf 

infarction and stenocardia, the abnormal movement of 
the cardiac wall is observed. It is desirable that 
such observation is conducted while parts other than 



a 

'Tl ^5 the to-be-observed heart are static. This is because 

it is necessary to find out an abnormally moving region 
within the incessantly moving cardiac wall and to make 
a precise diagnosis. 

Meanwhile, in case of an picture diagnosis by 

20 using the ultrasonic diagnosis apparatus, a system for 

collecting pictures by applying an ultrasonic probe to 
a subject is adopted. In this system, not only the 
patient but also the ultrasonic probe is not fixed and 
it is possible to freely change an observed region. 

25 Such a system is advantageous in -eonvenience and the 

degree of freedom. However, if the movement of the 
heart is observed using this system, pictures tend to 
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be blurred due to the movement of the subject's body 
and the movement of the inspector's hands. Even if 
pictures are collected while carefully preventing the 
blur of pictures, it is not clear whether or not 
5 pictures not blurred and suited for diagnosis can be 

obtained and the inspector needs to determine whether 
or not the pictures are appropriate. This determina- 
tion lacks objectivity and causes an increase in the 
I burden on the inspector. Such a system cannot expect 

10 effective diagnosis and may increase diagnosis time. 

As described above, if making a diagnosis of a 

'"4 

heart by using the ultrasonic diagnosis apparatus, it 
is necessary to pay attention to collecting blur-free 
'■J pictures suited for diagnosis. The determination 

M 15 whether blur- free pictures have been obtained depends 

on the inspector, which disadvantageously lacks 
objectivity and imposes a burden on the inspector. 

Moreover, since such information as to whether 
pictures useful for the diagnosis have been collected 
20 cannot be obtained, effective diagnosis cannot be 

conducted and more time is required for the diagnosis. 

Furthermore, if movements of ultrasonic pictures 
of the heart are analyzed on various points of the 
heart, they are considered beneficial to the various 
25 diagnoses of cardiac functions. To this end, if 

movement states of respective parts can be tracked 
by extracting contours of the pictures, states of 



functions of the parts of the heart can be properly 
grasped and more beneficial diagnosis can be expected. 

However, according to the conventional method of 
displaying contours of the heart obtained as ultrasonic 
pictures, the extracted contours are merely superim- 
posed over original pictures or displayed completely 
differently. Due to this, it is difficult to determine 
whether the extracted contours are correct or not and 
it is therefore impossible to track states of movements 
of various parts of the heart. It goes without saying 
that the conventional method cannot be applied to the 
grasp of more detailed states of cardiac functions. 

In addition, as software for drawing, coloring 
and filtering, there is computer graphic software 
(so-called a draw tool). Using the draw tool on a 
calculator, it is possible to form a graphic such as a 
contour manually and to modify, scale and color the 
manually formed contour. The draw tool, however, lacks 
functions such as comparison of a target and the 
picture of the target by superimposing the picture over 
the target and analysis of time series shape changes. 
Due to this, the draw tool cannot be applied to the 
evaluation of the accuracy of the contour extraction 
result and the analysis of the movement state of the 
moving target using contour lines~cf the picture of the 
target as described above. 

It is therefore an object of the present invention 



to provide an ultrasonic picture processing method and 
an ultrasonic picture processing apparatus capable of 
determining and collecting pictures which are important 
to the diagnosis of a disease and are not affected by 
5 the movement of the subject's body or the movement of 

inspector's hands, from ultrasonic pictures obtained by 
the ultrasonic diagnosis apparatus. 

Q 

.jg It is also another object of the present invention 

t. a 

'■'■4 

\l to provide an ultrasonic picture processing method and 

1=3 10 an ultrasonic picture processing apparatus capable of 

G easily determining whether extracted contours are 

correct, capable of tracking and analyzing movement 
:=^f states of various parts from the extracted contour 

'"- ^ 

xnformation and capable of making use of the result for 
■'^ 15 a diagnosis. 

It is another object of the present invention to 
provide a picture processing apparatus and a picture 
processing method capable of automatically calculating 
a terminal diastole period area or volume and a 
20 terminal systole period area or volume, capable of 

lessening a burden to the inspector and capable of 
obtaining an objective, accurate inspection result. 

It is another object of the present invention to 
provide a cardiac function analysis support apparatus 
25 and a cardiac function analysis support method of 

precisely associating cardiac wall contours in various 
time phases to allow information calculated from 



'J 
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contour information to be displayed in the form which 
can be easily evaluated and thereby allow local 
movement states of the cardiac wall to be easily 
evaluated . 

BRIEF SUMMARY OF THE INVENTION 
The present invention provides an ultrasonic 
picture processing method and an ultrasonic picture 
processing apparatus wherein contour information of a 
target is extracted from ultrasonic moving picture 



IS" 10 information, the contour information is sampled at 

preset time intervals for generating a plurality of 
'! time series contour data, and the contour data is 

^J? compared with other contour data, respectively to 

''''4 output comparison results. 

1— 15 The present invention provides an ultrasonic 

picture processing method and an ultrasonic picture 
processing apparatus wherein contour information of a 
target is extracted from ultrasonic moving picture 
information, the contour information is sampled at 

20 preset time intervals to generate a plurality of time 

series contour data, and the contour data is sequen- 
tially compared with other adjacent contour data, 
respectively, to make a predetermined determination of 
ultrasonic moving pictures based on comparison results 

25 The present invention provi'di^s a picture 

processing apparatus and a picture processing method 
having functions of extracting the contour of a target 
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on a picture, superimposing the extracted contour over 
the target on the picture and using, as the displayed 
contour, a dotted line or a contour line in at least a 
desired portion. 
5 The contour of the target on the picture is 

extracted. A display picture is generated by superim- 
posing the extracted contour, which is indicated by a 
g dotted line, over the target on the picture or by 

superimposing the extracted contour over the target on 
10 the picture only in a required region, and is then 

outputted. That is, the contour is indicated by a 
dotted line or part of the contour, such as an upper 
half of the contour and a right half of the contour, is 
displayed. In this way, it is possible to display the 
15 contour to the extent that the entire contour can be 

estimated only from information about the displayed 
portion. As a result, the shape of the extracted 
contour and that of the target can be easily compared. 
The present invention also provides a picture 
20 processing apparatus comprising means for operating a 

position command and a movement command and means for 
extracting the contour of the target on the picture, 
generating a display picture so as to display the 
extracted contour and the target by superimposing the 
25 extracted contour over the target on the picture and 

generating a display picture so as to temporarily 
change the shape of or temporarily move the position of 



fi.tr 
"shi 
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part of the contour including a portion indicated by 
the position command operation to thereby display the 
part of the contour together with the target on the 
picture . 

5 The contour of the target on the picture is 

extracted. The extracted contour is superimposed over 
the target on the picture to thereby become a display 
picture. In addition, by operating a position command, 
the desired position of the contour is indicated. By 

10 operating a movement command, the movement of the 

contour is indicated. In accordance with these 
commands, a display picture is changed such that the 
entire contour or part of the contour including a 
portion indicated by the command operation is temporar- 

15 ily deformed or moved. That is, if, for example, part 

of the contour is pulled by using a mouse, it is 
temporarily deformed and displayed. If stopping the 
mouse operation, part of the contour which has been 
pulled, returns to an original position and displayed. 

20 As a result, the shape of the extracted contour and 

that of the target can be easily compared. 

The present invention also provides a picture 
processing apparatus having a function of displaying 
the entire extracted contour or part of the extracted 

2 5 contour which is expanded outside— by a predetermined 

amount or contracted inside in accordance with the 
above command operation. 



If a command is issued by the command operation, 
the entire extracted contour or part of the extracted 
contour which is expanded outside by a predetermined 
amount or contracted inside by a predetermined amount 
is displayed. As a result, the shape of the extracted 
contour and that of the target can be easily compared. 

The present invention also provides a picture 
processing apparatus having a function of displaying 
the entire extracted contour region or part of the 
contour region by moving it horizontally or vertically 
by a predetermined amount in accordance with the above 
command operation. 

If a command is issued by the above command 
operation, the display picture is changed such that the 
entire extracted contour or designated part of the 
extracted contour, which is moved horizontally or 
vertically by a predetermined amount in accordance with 
the command operation, is displayed. Therefore, it is 
possible to manually change the position of the contour 
on the display picture while keeping the original shape 
and magnitude. As a result, the shape of the extracted 
contour and that of the target can be easily compared. 

The present invention further provides a picture 
processing apparatus having functions of extracting the 
contour of the target on the picture, oscillating and 
thereby displaying the entire extracted contour or part 
of the extracted contour externally, internally or both 



- 10 - 

externally and internally by a predetermined amount 
with reference to the contour position of the target on 
the picture. 

The contour of the target on the picture is 
5 extracted. The display picture is changed such that 

the entire extracted contour or part of the extracted 
contour is repeatedly moved so as to oscillate it 
externally, internally or both externally and inter- 

■ 'g nally by a predetermined amount with reference to the 

J' 10 contour position of the target on the picture. 

'"t Therefore, the wobbling (wavering) contour is displayed 

while maintaining the original shape, thereby facili- 

y tating comparing the shape of the extracted contour 

! 'sr 

^ with the shape of the target. 

'■•'=,3 

15 In addition, the present invention provides a 

picture processing apparatus comprising means for 
extracting the contour of the target on the picture and 
issuing a command for switching between a display and a 
non-display of the contour at predetermined time 

20 intervals and means for generating a display picture 

obtained by superimposing the contour over the target 
picture during a time period in which the contour is to 
be displayed and for generating a display picture 
including only the target picture during periods other 

25 than the above time period to thexeby flash-displaying 

the contour picture. 

The contour of the target on the picture is 



extracted. A display picture is generated by superim- 
posing the extracted contour over the target on the 
picture, whereby the contour is displayed. If a 
command is issued, the contour display period and the 
contour non-display period are switched. During the 
contour non-display period, the contour is eliminated 
and only the target picture is present on the display 
picture. During the contour display period, the 
display picture is changed to the picture where the 
contour is superimposed over the above picture, whereby 
the contour is displayed. As a result, if a command is 
issued, the contour is flash-displayed, thus facilitat- 
ing comparing the shape of the extracted contour with 
that of the target. 

The present invention provides a picture process- 
ing apparatus comprising means for extracting the 
contour of the target on the picture and issuing a 
switching command, means for selectively generating a 
display picture obtained by superimposing the contour 
of the target over the target picture and a display 
picture including only the target picture, in accor- 
dance with the switching command. 

The contour of the target on the picture is 
extracted. A display picture is generated by superim- 
posing the extracted contour over the target on the 
picture, whereby the contour is displayed. When a 
switching command is issued, the display picture is 
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changed to a display picture including only the target 
picture, and displayed. As a result, if a switching 
command is give by an operator's operation, it is 
possible to eliminate or display the contour by the 
5 operator's operation, thus facilitating comparing the 

shape of the extracted contour with that of the target. 
The present invention also provides a picture 
□ processing apparatus comprising means for extracting 

\\ the contour of the target on the picture and designat- 

f/l 10 ing a desired region of the extracted contour, means 



for issuing a switching command, means for selectively 
generating a display picture obtaining by superimposing 
the contour of the target over the target picture , a 
display picture including only the target, a display 

15 picture obtaining by eliminating the contour desig- 

nated by the designation means or the contour other 
than the designated contour from the superimposed 
picture of the target picture and the target contour, 
in accordance with the switching command. 

20 The contour of the target on the picture is 

extracted. A display picture is generated by superim- 
posing the extracted contour over the target on the 
picture, whereby the extracted contour is displayed. 
When a desired region of the extracted contour is 

25 designated and a switching commanxt is issued, the 

display picture is changed to a display picture 
obtained by superimposing the contour over the above 



target picture while the designated contour or the 
contour other than the designated contour is eliminated 
in accordance with the switching command, and the 
changed picture is displayed. When a switching command 
is given^ the display picture is changed to a display 
picture including only the target picture and displayed. 
As a result, if a switching command is given by the 
operator's operation, it is possible to eliminate or 
display the contour by the operator's operation, and to 
designate an eliminated region or a display region, 
thus facilitating comparing the shape of the extracted 
contour with that of the target. 

The present invention provides a picture process- 
ing apparatus having functions of extracting the 
contour of the target on the picture, making the 
brightness of the picture with that of the original 
picture and generating a display picture so as to 
display the contour and the original picture with 
different colors. 

The contour of the target on the picture is 
extracted. A display picture is generated by superim- 
posing the extracted contour over the target on the 
picture, and the display picture is displayed. At this 
time, the brightness of the contour is made propor- 
tional to that of the original picture and is given a 
color different from that of the original picture. By 
so doing, the contour and the original picture are 



displayed with difference colors. That is, a translu- 
cent contour is displayed. As a result, while the 
original picture can be observed even in the contour 
portion, the contour portion can be also observed, thus 
facilitating comparing the shape of the extracted 
contour with that of the target. 

As described above, the present invention is 
intended to extract the contour of the target on the 
picture, to divide the extracted contour into a display 
portion and a non-display portion and to display the 
target picture and the display portion of the contour 
by superimposing the display portion of the contour 
over the target picture. In addition, the present 
invention is intended to extract the contour of the 
target on the picture and to display the extracted 
contour region by superimposing it over the target on 
the picture and by scaling it. Furthermore, the 
present invention is intended to display the contour 
with a different color from that of the original 
picture and to switch the display picture between the 
picture obtained by superimposing the contour over the 
target and the picture including only the original 
picture. Therefore, the contour picture and the 
original picture can be easily compared, thereby making 
it easy to compare the shape of tire contour and that of 
the original picture at high accuracy. Due to this, it 
becomes possible to apply the present invention to the 



evaluation of the accuracy of the contour extraction 
result or the analysis of the movement state of the 
moving target by using the contour lines. 

The present invention provides a picture process- 
ing apparatus comprising means for extracting contours 
of the target from a series of moving pictures, means 
for obtaining contour internal areas for respective 
pictures from the contour information extracted from 
the respective pictures, means for detecting either a 
maximum contour internal area or a minimum contour 
internal area or both maximum and minimum contour 
internal areas within a predetermined time period and 
means for obtaining the contour internal area of the 
above picture from the detected value. 

The picture processing apparatus for extracting 
contours of the heart from a series of heart moving 
pictures showing the cross section of the heart 
comprises means for obtaining contour internal areas 
for respective pictures from the contour information 
extracted from the respective pictures., means for 
obtaining either a maximum or minimum contour internal 
area or both maximum and minimum contour internal areas 
among the contours of the respective pictures within a 
predetermined time period and means for specifying 
pictures corresponding to either the heart terminal 
diastole period or the heart terminal systole period, 
or both the heart terminal diastole period and the 



heart terminal systole period based on the detected 
values . 

That is, the present invention provides a picture 
processing apparatus for extracting contours of the 
target from moving pictures obtained by, for example, 
the ultrasonic diagnosis apparatus, calculating areas 
of the extracted contours and detecting the maximum and 
minimum areas as a terminal diastole area and a 
terminal systole area, respectively. 

The present invention provides a picture process- 
ing apparatus for extracting contours of the target 
from moving pictures obtained by, for example, the 
ultrasonic diagnosis apparatus, respectively, calculat- 
ing internal volumes of the extracted contours using 
the extracted contours, obtaining maximum and minimum 
volumes among the obtained contour internal volumes and 
detecting the maximum and minimum volumes as a terminal 
diastole volume and a terminal systole volume, respec- 
tively . 

The present invention provides a picture process- 
ing method comprising the steps of extracting contours 
of the target from moving pictures obtained by, for 
example, the ultrasonic diagnosis apparatus, obtaining 
an area/volume showing a minimum moving amount on time- 
by-time basis from moving amounts -obtained by using the 
extracted contours and detecting a maximum and a 
minimum area/volume as a terminal diastole area/volume 



and a terminal systole area/volume, respectively. 

According to the present invention, it is possible 
to accurately determine the heart terminal diastole 
period and the heart terminal systole period from 
concrete areas or volumes of the heart and to obtain 
the pumping function of the heart quantitatively. Thus 
objective measurement information useful to the 
diagnosis of the subject's cardiac function can be 
easily obtained while lessening the burden on the 
inspector. 

The present invention provides a cardiac function 
analysis support apparatus and a cardiac function 
analysis support method for dividing cardiac wall 
contours into a plurality of regions based on the 
characteristic points of the cardiac wall contours 
detected manually or automatically from the cardiac 
wall contour information extracted manually or auto- 
matically from time series heart pictures obtained from 
the picture diagnosis apparatus, associating the 
divided contour regions in different time phases with 
one another and outputting cardiac movement information 

By so doing, even if the cardiac wall moves in 
parallel to the contour tangent direction, characteris- 
tic points can be detected. As a result, it is 
possible to accurately associate characteristic points 
for time series pictures and to accurately associate a 
plurality of pictures of the cardiac wall portions 



divided based on the characteristic points. That is, 
it is possible to divide a cardiac wall region and to 
analyze the cardiac function conformable to the actual 
cardiac wall movement. 

According to the present invention, an annulus 
valva and a cardiac apex or a papillary muscle are set 
as characteristic points. By so doing, inappropriate 
association is prevented from occurring in the vicinity 
of valves or the cardiac apex, which has been a problem 
particularly, to the center line method. Besides, the 
use of the papillary muscle enables association by 
short axis pictures to be conducted accurately. 

According to the present invention, the divided 
cardiac wall contour regions are classified for 
positions at the cardiac wall, respectively. Among the 
classified cardiac wall contours, adjacent contours are 
displayed with different colors or different brightness 
By so doing, it is possible to display cardiac wall 
regions referred to as a front wall cardiac apex 
portion and a lower wall base portion with different 
colors, thereby facilitating grasping the state of the 
wall movements in respective cardiac wall regions. 

According to the present invention, a picture in a 
desired time phase serving as a reference time phase is 
automatically or manually set amonc time series heart 
pictures, the cardiac wall contours are divided, moving 
amounts of division points of the contours of the 



pictures in respective time phases from the correspond- 
ing division points of the contour of the picture in 
the reference time phase are calculated, respectively. 
The calculated moving amounts are displayed by at least 
one of numerical display, graph display or color 
display of the cardiac wall. By so doing, it is 
possible to easily discriminate a region having a large 
moving amount from a region having a small moving 
amount in the cardiac wall, thereby increasing the 
efficiency of the analysis of the cardiac function. 

According to the present invention, velocity 
information obtained from the ultrasonic diagnosis 
apparatus is classified for the divided points of the 
cardiac wall contours. Velocity statistic is calcu- 
lated for the classified cardiac wall positions, 
respectively and the calculated velocity statistic is 
displayed by at least one of numerical display, graph 
display and color display of cardiac wall. By so doing, 
it is possible to easily confirm velocity information 
of respective cardiac wall regions classified for the 
sake of a diagnosis, thereby increasing the efficiency 
of the analysis of the cardiac function. 

The present invention is characterized in that a 
dynamic range of the moving amounts of the cardiac wall 
division points is detected, display colors for 
displaying velocity information obtained from the 
ultrasonic diagnosis apparatus on the picture are 



allotted within the dynamic range. By so doing, even 
if the overall wall movement is small, it is possible 
to clearly display differences in moving amounts 
between parts of the cardiac wall. 

Additional object and advantages of the invention 
will be set forth in the description which follows, and 
in part will be obvious from the description, or may be 
learned by practice of the invention. The object and 
advantages of the invention may be realized and 
obtained by means of the instrumentalities and combina- 
tions particularly pointed out in the appended claims. 
BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING 

The accompanying drawings, which are incorporated 
in and constitute a part of the specification, illus- 
trate presently preferred embodiments of the invention, 
and together with the general description given above 
and the detailed description of the preferred embodi- 
ments given below, serve to explain the principles of 
the invention. 

FIG. 1 is a block diagram showing an example of 
the structure of an ultrasonic picture processing 
apparatus using the ultrasonic picture processing 
method in the first embodiment according to the present 
invention; 

FIG. 2 is a flowchart for desncribing processing 
steps of the ultrasonic picture processing method 
in the first embodiment according to the present 



invention; 

FIG. 3 shows an example of an extracted cardiac 
wall contour; 

FIG. 4 is a diagram for describing a method for 
moving information about the extracted time series 
contours ; 

FIG. 5 shows an example of comparison results of 
moving information about sampled contours; 

FIG. 6 shows an example of moving information 
about cardiac wall contours without an influence of the 
movement of hands and the movement of "a body; 

FIG. 7 is a flowchart for describing processing 
steps of the ultrasonic picture processing method in an 
embodiment according to the present invention; 

FIG. 8 is a flowchart for describing processing 
steps of the ultrasonic picture processing method in an 
embodiment according to the present invention; 

FIG. 9 is a flowchart for describing recording 
processing of time series information for every 
sampling time and comparison processing of the recorded 
contour information in the ultrasonic picture process- 
ing method in an embodiment according to the present 
invention; 

FIG. 10 is a diagram for describing a method for 
dividing contours ; 

FIG. 11 is a diagram for describing a method for 
sampling divided time series contour information; 



FIG. 12 is a flowchart for describing processing 
steps of the ultrasonic picture processing method in an 
embodiment according to the present invention; 

FIG. 13 shows an example of a cardiac muscle 
region used for the extraction of moving information; 

FIG. 14 shows an example of a region used for the 
extraction of moving information; 

FIG. 15 is a flowchart for describing processing 
steps of the ultrasonic picture processing method in an 
embodiment according to the present invention; 

FIG. 16 shows an example of characteristic 
points on a picture used for the extraction of moving 
information in an embodiment according to the present 
invention ; 

FIG. 17 shows an example of a concerned region on 
a picture used for the extraction of moving information 
in the ultrasonic picture processing method in an 
embodiment according to the present invention; 

FIG. 18 is a schematic block diagram of the 
picture processing apparatus in an embodiment according 
to the present invention; 

FIG. 19 is a block diagram of the picture input 
section of the picture processing apparatus of FIG. 18; 

FIG. 20 is a block diagram of the contour 
extraction section of the picturei^rocessing apparatus 
of FIG. 18; 

FIGS. 21A and 2 IB show a contour region and 



a display region in the picture processing apparatus of 
FIG. 18, respectively; 

FIG. 22 shows an example of picture display in the 
picture processing apparatus of FIG. 18; 

FIG. 2 3 is a block diagram of the picture 
processing apparatus in an embodiment according to the 
present invention ; 

FIG. 24 is a diagram for describing an example of 
an enlarged contour in the picture processing apparatus 
of FIG. 23; 

FIG. 2 5 shows the operation input section of the 
picture processing apparatus of FIG, 23; 

FIG. 26 is a diagram for describing an operation 
example in the picture processing apparatus of FIG. 23; 

FIG. 2 7 is a diagram for describing an operation 
example in the picture processing apparatus of FIG. 23; 

FIG. 28 is a diagram for describing an operation 
example in the picture processing apparatus of FIG. 23; 

FIG. 29 is a diagram for describing an operation 
example in the picture processing apparatus of FIG. 23; 

FIG. 30 is a block diagram of the picture 
processing apparatus in an embodiment according to the 
present invention; 

FIG. 31 is a block diagram of the picture 
processing apparatus in an embodiment according to the 
present invention ; 

FIG. 32 is a block diagram of the picture 



processing apparatus in an embodiment according to the 
present invention; 

FIG. 33 is a flowchart for showing an example of a 
picture processing flow in the picture processing 
apparatus of FIG. 32; 

FIG. 34 shows an example of an extracted heart 
contour in the picture processing apparatus of FIG. 32; 

FIG. 35 is a flowchart for showing a detailed 
processing flow of the processing step S204 of FIG. 33; 

FIG. 36 is a block diagram of the picture 
processing apparatus in an embodiment according to the 
present invention; 

FIG. 37 is a flowchart showing an example of a 
picture processing flow in the picture processing 
apparatus of FIG. 36; 

FIG. 38 is a diagram for describing how to obtain 
an area by counting the number of pixels in the picture 
processing of FIG. 36; 

FIG. 39 is a block diagram of the picture 
processing apparatus in an embodiment according to the 
present invention ; 

FIG. 40 is a flowchart for showing a processing 
flow in the picture processing apparatus in the 
embodiment of FIG. 39; 

FIG. 41 is a diagram for describing how to obtain 
an area by polygon approximation in the processing of 
FIG. 40; 



FIG. 4 2 is a block diagram of the picture 
processing apparatus in an embodiment according to the 
present invention; 

FIG. 43 is a flowchart showing a processing flow 
in the picture processing apparatus in the embodiment 
of FIG. 42; 

FIG. 44 is a diagram for describing how to obtain 
a volume of the cross section of the heart; 

FIG. 45 is a block diagram of the picture 
processing apparatus in an embodiment according to the 
present invention ; 

FIG. 46 is a flowchart showing a processing flow 
in the picture processing apparatus in the embodiment 
of FIG. 45; 

FIG. 47 is a block diagram of the picture 
processing apparatus in an embodiment according to the 
present invention ; 

FIG. 48 is a flowchart showing a processing flow 
in the picture processing apparatus in the embodiment 
of FIG. 47; 

FIG. 49 is a block diagram of the picture 
processing apparatus in an embodiment according to the 
present invention; 

FIG. 50 is a flowchart showing a processing flow 
in the picture processing apparatus in the embodiment 
of FIG. 49; 

FIG. 51 is a block diagram of the cardiac function 



analysis support apparatus in an embodiment according 
to the present invention; 

FIG. 52 is a flowchart for showing a processing 
flow in the cardiac function analysis support apparatus 
of FIG. 51; 

FIG. 5 3 is an explanatory diagram of the inputted 
contour in the embodiment of FIGS. 52 and 53; 

FIG. 54 is an explanatory diagram of the detection 
of characteristic points in the embodiment of FIGS. 52 
and 53; 

FIG. 55 is an explanatory diagram of contour 
division in the embodiment of FIGS. 52 and 53; 

FIG. 56 is an explanatory diagram of association 
of cardiac wall contours in a plurality of time phases 
in the embodiment of FIGS. 51 and 52; 

FIG. 57 is an explanatory diagram of a display 
method in the embodiment of FIGS. 51 and 52; 

FIG. 58 is an explanatory diagram of a display 
method in the embodiment of FIGS. 51 and 52; 

FIG. 59 is a diagram for describing an example of 
applying the embodiment of FIGS. 51 and 52 to a short 
axis heart picture; 

FIG. 60 is a block diagram of the cardiac function 
analysis support apparatus in an embodiment according 
to the present invention; 

FIG. 61 is a flowchart for describing a processing 
flow of the cardiac function analysis support method in 



the embodiment of FIG. 60; 

FIG. 62 is an explanatory diagram of a display 
method in the embodiment of FIGS. 60 and 61; 

FIG. 63 is an explanatory diagram of a method for 
displaying velocity change in the embodiment of 
FIGS. 60 and 61; 

FIG. 64 is an explanatory diagram of a movement 
area in the embodiment of FIGS. 60 and 61; 

FIG. 65 is a block diagram of the cardiac function 
analysis support apparatus in an embodiment according 
to the present invention; 

FIG. 66 is a flowchart for describing a processing 
flow of the cardiac function analysis support method in 
the embodiment of FIG. 65; 

FIG. 67 is an explanatory diagram of a display 
method in the embodiment of FIGS. 65 and 66; 

FIG. 68 is a block diagram of the cardiac function 
analysis support apparatus in an embodiment according 
to the present invention; and 

FIG. 69 is a flowchart for describing a processing 
flow of the cardiac function analysis support method in 
the embodiment of FIG. 68. 

DETAILED DESCRIPTION OF THE INVENTION 

Embodiments of the present invention will be 
described with reference to the drawings. 

First, the structure of an ultrasonic picture 
processing apparatus will be described with reference 
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to FIG. 1. 

In FIG. 1, an ultrasonic diagnosis apparatus 10 is 
connected to an ultrasonic picture processing apparatus 
20. The ultrasonic picture processing apparatus 20 
5 comprises a moving picture input section 21, a contour 

extraction section 22, a sampling section 23, a storage 
24, a comparator 25 and an output section 26. Heart 
p moving pictures outputted from the ultrasonic diagnosis 

□ apparatus 10 are sequentially inputted into the moving 

i;g 10 picture input section 21. The contour extraction 

section 22 extracts contours of moving pictures 
ir^P^tted into the moving picture input section 2 1 or 
i'jj extracts a region of the heart. 

;:i An information divider 2 3 divides time series 

'^j 15 information of contours of a cardiac wall or a region 

of the heart extracted at the contour extraction 
section 22 for each predetermined sampling time period, 
for example, a sampling time period set based on a 
heart rate measurement value inputted from an external 
20 apparatus. The divided information is temporarily 

stored in the storage 24. 

The comparator 25 compares the divided information 
about the contours of the cardiac wall or the regions 
of the heart stored in the storage 24 with one another. 
25 The comparator 25 also compares a comparison value (or 

a reference value) previously stored therein and suited 
for a diagnosis with the divided information about the 
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contours of the cardiac wall or the regions of the 
heart stored in the storage 24, to determine whether 
moving pictures are useful for the diagnosis. The 
moving pictures, if determined as useful pictures, are 
5 stored in, for example, the storage 24, 

The output section 26 includes, for example, a 
display for displaying comparison values indicated by a 
□ formula (1) in a time series manner; i.e., Diff (a, b) , 

'^-1 Diff (b, c), ... or as shown in FIG. 5. The output 
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10 section 26 also reads the moving pictures determined as 

useful for the diagnosis by the comparator 25, from the 
storage 24 and displays them. The moving pictures 
determined as useful pictures are fed to the ultrasonic 
diagnosis apparatus 10 to serve as processing targets 
15 during predetermined diagnosis processing. 

FIG. 1 describes the ultrasonic picture processing 
apparatus 20 as an external apparatus connected to the 
ultrasonic diagnosis apparatus 10, The structure 
elements of the ultrasonic moving picture processing 
20 apparatus 2 0 shown in FIG. 1 may be incorporated into 

the ultrasonic diagnosis apparatus 10. 

The ultrasonic moving picture processing method in 
the first embodiment according to the present invention 
will be described with reference to FIGS. 1 through 6, 
using a case of the diagnosis of the heart by way of 
example . 

As shown in the flowchart of FIG. 2, heart moving 
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pictures are inputted into the moving picture input 
section 21 of the ultrasonic moving picture processing 
apparatus 20 from the ultrasonic diagnosis apparatus 10 
(SI). Contours of a target (cardiac wall) are ex- 
tracted from the inputted moving pictures by the 
extraction unit 22 (S2). The extraction of the 
contours of the cardiac wall can be automatically and 
easily performed by using, for example, a method of 
active contour models and balloons (Laurent D, Cohen, 
CVGIP: lU, 53(2): 211=218, 1991). Next, sampl ing time 
periods for dividing time series contour information 
are set (S3). In this embodiment, the values which 
have been inputted in advance are set as sampling time. 
Thereafter, respective pieces of time series contour 
information extracted at the thus set sampling time are 
stored in the storage 24 (S4). Pieces of the stored 
contour information are compared with one another by 
the comparator 25 (S5). 

The comparison in this embodiment is conducted as 
follows. As shown in FIG. 3, movement amounts of the 
entire cardiac wall contour 30 extracted in the step S2 
are averaged and divided for every sampling time St and 
the movements are recorded (see FIG. 4). In this 
embodiment, since absolute values of the movement 
amounts are recorded with reference to a contour at a 
given time period, positive values are obtained as 
shown in FIG. 4. Due to the periodic cardiac movement. 
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a periodic graph is obtained as shown in FIG. 4. 

The sampled data series are set as Ta, Tb, Tc ... 

and adjacent data series such as Ta and Tb, Tb and Tc , 

are compared. The comparison is made using an error of 

5 mean square as shown in the following formula (1): 

r^.^x. 1.. V(Ta(i) - Tb(i) )2 
Dxff (a,b) = 2_, — ^—LJ— (1) 

x = 0 

The formula (1) shows an example of the comparison 
of the data series Ta and Tb. In the formula (1), 
Ta(i) denotes the i-th data (i-th movement amount) of 

10 the data series Ta, Tb(i) denotes the i-th data of the 

data series Tb and n denotes the number of data series . 
The number n is determined by the processed number 
during the extraction of the cardiac wall contours. 
For example, if the frame rate of moving pictures 

15 obtained from the ultrasonic diagnosis apparatus is 30 

(frame/second) and contours of all of the frames of the 
moving pictures are extracted, then n is calculated by 
the following formula (2), while the contour data 
sampling time set as Ts (second) as described above: 

20 n = Ts X 30 (2) 

Finally, comparison values obtained from the 
formula (1) are displayed in a time series manner such 
as Diff (a, b) , Diff (b, c), ... as shown in FIG. 5 
(S6). 

Due to the periodic heart movement, if there are 
no movements, such as the movement of the body or the 
movement of hands, other than the heart , a periodic 
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and uniform curve is provided by plotting the movement 
amounts of the cardiac wall as shown in FIG. 6. In 
such a case, the comparison value Diff in the formula 
(1) is quite low (almost "0"). 
5 On the other hand, if the movement of the body or 

the movement of hands are added to the movement of the 
cardiac wall, the movement amounts of the cardiac wall 
are expressed as an irregular curve as shown in FIG. 4 
and the comparison value Diff becomes higher. As such 

10 a manner, since it is possible to determine whether or 

not the collected moving pictures include the movement 
of the body or the movement of the hands by comparing 
moving amounts of the contours in respective sampling 
periods, it is possible to easily collect moving 

15 pictures useful for the diagnosis. 

In this embodiment, comparison values of the 
contour moving amounts are displayed for determining 
whether or not the moving pictures are useful for the 
diagnosis. It is also possible to determine the 

20 usefulness of the moving pictures by displaying graphs 

obtained by plotting contour movements on time-by-time 
bases as shown in FIGS. 4 and 6. In the latter case, 
determination cannot be made based on a quantitative 
index. However, since visual determination is possible 

25 to a certain extend, pictures f ree^ f rom the movement of 

the body or the movement of the hands can be easily 
collected. The comparison result of contour movement 



amounts are displayed by plotting comparison values on 
time-by-time basis. The comparison values may be 
displayed directly . 

Next, the second embodiment according to the 
present invention will be described with reference to 
FIG. 7. 

FIG. 7 is a flowchart showing other processing 
steps of the ultrasonic picture processing method in 
this embodiment. The same steps as those in FIG. 2 are 
denoted by the same reference numerals. Description 
will be given only to steps different from the case of 
FIG. 2. That is, in this embodiment, a comparison 
value suited for a diagnosis is set as a reference 
value in advance as shown in FIG. 7. By comparing the 
contour information with the reference value, it is 
determined whether or not moving pictures are useful 
for the diagnosis (S7). The comparison results are 
displayed, as well (S6). 

The ultrasonic moving pictures determined as 
useful for the diagnosis are fed back to, for example, 
the ultrasonic diagnosis apparatus and become diagnosis 
targets . 

The third embodiment according to the present 
invention will be described with reference to FIG. 8. 

FIG. 8 is a flowchart showing other processing 
steps of the ultrasonic picture processing method in 
this embodiment. In these steps, sampling time is 



determined based on heartbeat information obtained by 
an electrocardiograph as shown in the block diagram of 
FIG. 1. 

Specifically, as shown in FIG. 8, the processing 
is the same as that of FIG. 2 from steps SI through S6 . 
However, in step S3, when sampling time is set, 
heartbeat is measured by the electrocardiograph (S8) 
and sampling time is set based on the heartbeat 
information obtained from the electrocardiograph. 

Due to the periodic cardiac movement, it is 
desirable that the sampling time for time series 
contour information is the same as the period of the 
heartbeat. The heartbeat period of a human is about 1 
second; however, it depends on individual subjects. 
Therefore, if setting sampling time based on the 
information from the electrocardiograph as mentioned 
above, it is possible to determine whether or not the 
collected moving pictures are suited for the diagnosis 
at high precision. 

Next, another embodiment will be described with 
reference to FIGS. 9 through 11. This embodiment 
illustrates a case of calculating comparison values not 
using an average contour movement amount but the 
contour movement amounts of respective parts. Descrip- 
tion will be given only to elements- different from 
those of the preceding embodiments. 

As shown in FIG. 9, in step S4, when time series 
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contour information is recorded, the extracted cardiac 
wall contour 30 are divided into a plurality of regions 
(Rl, R2 , RN-1), respectively as shown in FIG. 10 

(S9). Time series information about respective divided 
5 regions (such as regions RO, Rl) is recorded for every 

sampling time (St) as shown in FIG. 11 (SIO) . 

Next, during the comparison of contour information 
in the step S5, data series comparison is made for 
respective divided regions (RO, Rl, RN-1) by using 

10 the above-described method and comparison values in the 

respective regions are calculated by the following 
formula (3) (Sll) . 

Diff_RO(a,b) = "^W(Ta(i)-Tb(i))2^ 

1 = 0 



Dif£_Rl(a,b, = "^W(Ta(i)-Tb(i)^ 



i = 0 



In the formula (3), DiffARO (a, b) denotes a 
15 comparison value of the data series Ta and Tb in the 

region RO and Diff ARl (a, b) denotes a comparison 
value of the data series Ta and Tb in the region Rl . 
Thereafter, in this embodiment, the calculated compari- 
son values are averaged using a formula (4) (S12) and 
20 the results are displayed as comparison values of the 

overall contours as shown in FIG. 5 (S6). 
N-1 

^ Dif f _ Rl(a,b) 
1=0 

(4) 

N 



In the formula (4), N denotes the number of 
divided contours. Other processing steps are the same 
as those in the first embodiment. 

Yet another embodiment according to the present 
invention will be described with reference to FIGS. 12 
and 13. This embodiment illustrates a case of using 
region information as time series data used in the 
comparison processing in the embodiment of FIG. 2. 

FIG. 12 is a flowchart showing the processing 
steps of the ultrasonic picture processing method of 
this embodiment. 

Heart moving pictures are inputted from the 
ultrasonic diagnosis apparatus (531). A region of the 
heart is extracted from the inputted moving pictures 
(S32). In this embodiment, the region refers to a 
heart muscle (a region between an outer cardiac wall 31 
and an inner cardiac wall 32). 

Thereafter, as in the case of FIG. 8, the heart 
rate is measured by the electrocardiograph (S38) and 
sampling time is set based on the information obtained 
from the measurement of the heart rate (S33). Time 
series region information is recorded for every 
sampling time (S34). Respective pieces of the recorded 
time series region information are compared (S35). 
Finally, comparison results are -drsplayed (S36). 

In this embodiment, the region information used in 
the comparison is related to the region of the entire 



heart muscle. However, it is also possible to divide 
the heart muscle into a plurality of regions as 
described in the embodiment of FIG. 9 and to compare 
time series region information. Furthermore, this 
embodiment uses the heart muscle as a target region. 
However, using the fact that the echo strength of blood 
and those of other portions differ (usually, the echo 
strength of blood is lower than those of other por- 
tions) in the ultrasonic diagnosis apparatus, it is 
possible to divide a target region into a blood region 
41 surrounded by the cardiac wall 40 and the remaining 
region (indicated by oblique lines) as shown in FIG. 14 
and to obtain movement information about one of or both 
of the divided regions . 

Another embodiment will be described with 
reference to FIG. 15. This embodiment illustrates a 
case of conducting processing (recording of moving 
pictures) in accordance with the result of the recorded 
time series information. In the flowchart of FIG. 15, 
the same processing steps as those of FIGS. 2, 7 and 8 
are denoted by the same reference numerals. 

In FIG. 15, after comparing the recorded contour 
information in a step S5, the results are displayed in 
a step S6. As described in the embodiment of FIG. 2, 
it is determined whether or not a picture satisfies a 
preset criteria (whether it is a picture free from the 
hand movement or the body movement and suited for the 



diagnosis) (S7). If satisfying the criteria, the 
original moving picture is recorded in a predeteinnined 
storage medium (513). 

The moving pictures satisfying the criteria are 
fed back to, for example, the ultrasonic diagnosis 
apparatus and become diagnosis targets. 

By so doing, unnecessary pictures unsuited for the 
diagnosis are not recorded and wastefulness of the 
storage medium can be thus eliminated. 

The embodiments described with reference to 
FIGS. 1 through 15 employ the movement amounts of the 
contours or region of a target as information to be 
used when determining whether pictures are useful for 
the diagnosis. It is possible to employ movement 
amounts obtained by automatically detecting or manually 
setting characteristic points 50 on the target 40, i.e., 
adjacent to the contour thereof and tracking these 
movements . 

Alternatively, as shown in FIG. 17, it is possible 
to set an optional concerned region 60 on the target 
(or cardiac wall) 40, to track the region 60 by 
calculating an optical flow in the inner region or by 
using the correlation method and to thereby obtain and 
compare time series movement amounts. 

Although the above-described - embodiments employ a 
heart as a target, the present invention can be applied 
to organs other than the heart. In such cases, other 



organs do not move periodically like the heart and 
movements of the contours or region are not represented 
by a periodic curve. Due to this, it is not necessary 
to set sampling time periods based on the electrocar- 
diograph information. Rather, preset optional time 
periods can be used. 

The methods described above can be stored as 
programs which can be executed by a computer, in a 
storage medium such as a magnetic disc (floppy disc, 
hard disc and the like), an optical disc (CD-ROM, DVD 
and the like) and a semiconductor memory. 

In the above-described embodiments, contour 
information or region information is extracted from 
ultrasonic moving pictures and the time series contour 
information or region information is divided based on 
the preset sampling time periods. However, it is also 
possible to divide the ultrasonic moving picture based 
on preset sampling time periods in advance and then to 
extract contour information or region information from 
the frame pictures during the respective time period. 

As described above, according to the embodiments 
of the present invention, moving pictures which are 
free from the movement of the subject's body or the 
movement of the inspector's hands and important to the 
diagnosis of a disease can be easily determined and 
collected from the ultrasonic pictures obtained by the 
ultrasonic diagnosis apparatus. 



Now, the description will be given to a picture 
processing apparatus and method capable of displaying 
pictures such that it can be easily determined whether 
the extracted contours are correct or not and that 
movement states of respective portions can be tracked 
and analyzed from the extracted contour information. 

As shown in FIG. 18, the picture processing 
apparatus comprises a picture input section 101, a 
contour extraction section 102, a display region 
determination section 103 and a display picture 
generation section 104. 

The picture input section 101 receives ultrasonic 
pictures from, for example, the ultrasonic diagnosis 
apparatus, as original picture and input the pictures 
into the contour extraction section 102. The contour 
extraction section 102 has a function of extracting 
contours of a target from the original pictures 
inputted by the picture input section 101. The display 
region determination section 103 has a function of 
determining a region to be displayed from the contour 
region (the entire region of the extracted contour 
lines) extracted by the contour extraction section 
102. The display picture generation section 104 has 
a function of generating a display picture by superim- 
posing a display region within the-eontour region 
determined by the display region determination section 
103 over the original pictures previously inputted by 
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the picture input section 101. The picture display 
section 105 displays the picture obtained by the 
display picture generation section 104. 

With the above structure, when the picture input 
5 section 101 inputs an original picture such as an 

ultrasonic picture into the contour extraction section 
102, the contour extraction section 102 extracts the 
contour of a target from the original picture received 
^ from the picture input section 101 by a method using 

10 active contour models and feeds the contour to the 

display region determination section 103. The display 
^^gion determination section 103 determines a display 
;f! region from the extracted contour region (the entire 

^•^^ region of the extracted contour lines) and feeds the 

-J 

15 ' determined information, that is, the display region to 
the display picture generation section 104, The 
display picture generation section 104 generates a 
display picture by superimposing the display region 
within the contour region determined by the display 
20 region determination section 103 (a display target 

region within the entire region of the contour lines) 
over the original picture previously inputted by the 
picture input section 101. The picture display section 
105 displays the display picture obtained by the 
25 display picture generation section 104, that is, a 

superposition picture of the contour picture over the 
original picture. The display picture is therefore 



a picture obtained by superimposing the respective 
contour lines over the original picture, whereby both 
the original picture and the contour lines can be 
observed. This makes it possible to easily verify 
whether the contour lines are aligned to the target 
portion and to observe a picture. 

The picture input section 101 has a structure as 
shown in FIG. 19. Namely, the picture input section 
101 comprises an picture pickup unit 106, an picture 
data converter 107 and a picture storage section 108. 

Among these elements, the picture pickup unit 106 
obtains a real-time moving picture (as well as a still 
picture). The unit 106 corresponds to, for example, a 
medical diagnosis apparatus such as an ultrasonic 
diagnosis apparatus. The picture data converter 107 
converts a picture signal obtained by the picture 
pickup unit 106 into a data format in units of frames 
(units of picture planes) so that the contour extrac- 
tion section 102 can handle it. The picture storage 
section 108 stores picture signal data converted by the 
picture data converter 107, 

The picture input section 101 having this 
structure coverts a picture signal from the picture 
pickup unit 106, such as an ultrasonic diagnosis 
apparatus, into a data format which can be handled by 
the contour extraction section 102 at the picture data 
converter 107 and stores the converted data in the 



picture storage section 108. The picture input section 
101 then reads the data stored in the picture storage 
section 108 and supplies the data to the contour 
extraction section 102 and to the display picture 
generation section 104. 

The contour extraction section 102 has a structure 
as shown in FIG. 20. A case of extracting contours 
using active contour models will be described hereinaf- 
ter. The contour extraction section 102 comprises an 
initial value setting section 109, an energy calculator 
110, a convergence determination section 111 and a 
discrete point moving section 112. 

Among these elements, the initial value setting 
section 109 has a function of setting an initial 
contour. In this case, the initial value setting 
section 109 sets a contour by manually providing the 
coordinate of the initial contour. If the coordinate 
of the initial contour is provided manually, a GUI 
(graphical user interface) and a pointing device such 
as a mouse, a pen, a track ball, are used. By operat- 
ing the direction cursor of the pointing device, a 
coordinate is designated on the display picture plane. 
An initial contour is thereby plotted and inputted on 
the picture plane. An original picture is displayed on 
the display plane. Using the pointing device, the 
coordinates of positions of the contour lines are 
designated and plotted on the original picture. The 



inputted coordinate positions are displayed on the 
picture plane and stored as coordinate position 
information. The contour thus inputted is expressed as 
a collection of representative coordinates on the 
contour and stored in this form in the built-in memory 
of the initial setting section 109. 

The energy calculator 110 calculates the sum of 
the internal energy of the contour, picture energy and 
external energy given at need. The discrete point 
moving section 112 has a function of moving the contour 
(conducting a convergence calculation) so that the 
energy sum calculated by the energy calculator 110 is 
as small as possible. 

The convergence determination section 111 
functions to receive information about the sum of 
energies calculated by the energy calculator 110, to 
determine whether or not a change in the sum of energy 
is smaller than a predetermined value, to determine 
that the sum of energy is convergent when the change is 
smaller than the predetermined value, to issue a 
command to stop the convergence calculation to the 
energy calculator 110 and to thereby end the conver- 
gence calculation at the energy calculator 110. 

The display region determination section 103 
functions to set a region for comrecting representative 
points on the contour obtained by the contour extrac- 
tion section 102 by straight lines or curves and to 



determine a display portion within the region. 

The display picture generation section 104 
functions to replace only the pixel value in the 
portion of the original picture inputted by the picture 
input section 101 corresponding to the display region 
within the contour region determined by the display 
region determination section 103, with, a different 
pixel value and to thereby generate a display picture 
having the display region within the contour region 
superimposed over the original picture. The picture 
display section 105 functions to display the picture 
generated by the display picture generation section 104 
on a display device. 

The contour extraction section 102 having the 
above structure operates as follows. An initial 
contour is set by the initial value setting section 109. 
Coordinates of the initial contour are given manually. 
The contour is expressed as a collection of representa- 
tive coordinates on the contour and stored in this form 
in the built-in memory. When the initial contour is 
set by the initial value setting section 109, the 
energy calculator 110 calculates a sum of the internal 
energy of the initial contour, picture energy and 
external energy given at need. The discrete point 
moving section 112 moves the contour so that the energy 
sum calculated by the energy calculator 110 is as small 
as possible. 



The convergence determination section 111 monitors 
a change in the energy sum calculated by the energy 
calculator 110, determines whether this change is 
smaller than a predetermined value and issues a command 
to stop a convergence calculation to the discrete point 
moving section 112 when the change is smaller than the 
predetermined value. When the change is larger than 
the predetermined value, the convergence determination 
section 111 operates so that the discrete point moving 
section 112 repeats moving the contour. When the 
change is smaller than the predetermined value as a 
result, the discrete point moving section 112 ends 
moving the discrete point. 

The display region determination section 103 sets 
a display region as a region obtained by connecting 
representative points of the contour obtained by the 
contour extraction section 2 by straight lines or 
curves. The display region determination section 103 
determines a portion to be displayed in that region. 
The portion is, for example, a portion in which the 
contour is repeatedly displayed and not displayed at 
certain intervals (refer to FIGS. 21A and 21B). 

The display picture generation section 104 
generates a display picture by replacing only the pixel 
value of a portion of the origina-t picture inputted by 
the picture input section 101, corresponding to the 
display region within the contour determined by the 
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display region determination section 103, with a 
different pixel value. 

The display section 105 displays the picture 
generated by the display picture generation section 104 
5 on a display device such as a display (refer to 

FIG. 22) . 

Both the information about the contour extraction 
result and the information about the vicinity of the 
contour of the original picture can be indicated at a 
time by failing to display part of the contour region 
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□ as mentioned above. 

; in 



'J In other words, in this embodiment, when an 

□ original picture, which is, for example, an ultrasonic 

m 

□ picture of the heart, is inputted, a contour is roughly 

□ 15 inputted on the ultrasonic picture of the heart and the 
' inputted contour line is automatically corrected by the 

energy calculation and the like based on the inputted 
contour. Then the contour lines or the picture of the 
contour region in the designated display target portion 

20 is superimposed over the original picture and displayed. 

Furthermore, the picture on the displayed contour line 
or in the contour region is periodically replaced with 
the original picture, thereby making it possible to 
clearly compare the contour with the target portion of 

25 the original picture. 

The apparatus in this embodiment can easily 
extract a target portion of the original picture and 



allows a user to easily determine whether or not the 
extracted contour is correct based on the direct 
comparison of the extracted contour with the original 
picture . 

Therefore, by evaluating the accuracy of the 
extracted contour and utilizing the information about 
the accurately extracted contour for an analysis, it is 
possible to apply the contour line of the moving target 
picture to the analysis of the moving state of the 
target . 

Another embodiment for extracting and displaying 
contour lines will be described. 

In this embodiment, an example of displaying the 
contour expanded outside by a predetermined amount or 
reduced inside by means of inputting commands by a user 
interactively . 

The structure of this embodiment is illustrated by 
FIG- 23. The picture processing apparatus of FIG. 23 
comprises a picture input section 101,- a contour 
extraction section 102, a display region determination 
section 103, a display picture generation section 104, 
a picture display section 105 and an operation input 
section 113. 

The picture input section 101, the contour 
extraction section 102, the display picture generation 
section 104 and the picture display section 105 in this 
embodiment are the same as those in the embodiment of 



FIG. 18. The apparatus in this embodiment is charac- 
terized by comprising the operation input section 113. 
The operation input section 113 designates the picture 
display magnification of a target portion on the 
display plane, displays the target portion in the form 
shown in FIG. 25 on the picture plane and designates a 
scaling rate by selecting the rate using a mouse cursor. 

As shown in FIG. 25, the operation input section 
113 has three operations buttons; i.e., an "enlarge- 
ment" button 113a, a "reduction" button 113b, and a 
"return" button 113c. When pushing the "enlargement" 
button 113a, the picture scaling rate a is increased. 
When pushing the "reduction" button 113b, The picture 
scaling rate a is reduced. When pushing the "return" 
button 113c, the picture scaling rate a is returned to 
the original rate and a command to display a picture at 
100% magnification can be issued. This command is 
outputted by using the value of the scaling rate a . 

This embodiment shows an example of using buttons 
at the operation input section 113. A slide bar, a 
dial, a mouse and the like may be used as input means, 
as well. 

The display region determination section 103 
converts coordinate of the contour data X into X' based 
on the scaling rate a given from the operation input 
section 113 (refer to FIG. 24): 

X = {(xl, yl), (x2, y2),..., (xn, yn)} (5) 
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X = {(xl', yl'), (x2', y2 '),..., (xn', yn')} (6) 
Here, xi ' = xO + a ( xi - xO ) ; yi' = yO + a (yi - 
yO); n is the number of representative points; and (xO, 
yO) is the center of gravity of the contour. 
1 " 

5^0 = - E^i (7) 
^i = l 

1 " 

YO = - Z yi ( 8 ) 

"i = l 

The example of the operation will be described 
hereinafter . 

FIG. 26 illustrates a picture plane displayed on 
the picture display section 105. The picture display 
shows a state right after the contour extraction 
section 102 conducts processing shown in FIG, 18, the 
display picture generation section 104 generates a 
15 display picture obtained by superimposing an extracted 

contour region (all of the extracted contour lines) 
over an original picture, and the picture display 
section 105 displays the display picture 114. On this 
plane, reference numeral 114 denotes a picture of the 
20 target region obtained by superimposing the contour 

region over the original picture and reference numeral 
15 denotes buttons 113a to 113c. 

FIG. 27 illustrates a display- plane on the picture 
display section 105 when operating the buttons and, in 
25 particular, right after pushing the "enlargement" 

button 113a. Only the contour region (or extracted 



contour lines) is enlarged and the picture of the 
target region is not enlarged. By expanding only the 
contour region (or extracted contour lines) outside, 
the original picture in the vicinity of the contour 
region can be observed. Thereafter, by pushing the 
"return" button 113c, the picture returns to its 
original unsealed (in FIG. 28). If the contour region 
can be more easily observed by reducing the picture 
inside, the "reduction" button 113b is pushed to 
thereby reduce the picture inside (in FIG. 29). 

This embodiinent displays the picture in the 
contour region (or extracted contour lines) by scaling 
the picture on the original picture. Due to this, it 
is possible to easily compare the picture in the 
contour region with the original picture, thereby 
facilitating the comparison and determination of the 
shape at high accuracy. 

The embodiment shown in FIG. 30 illustrates a case 
of displaying the contour by wobbling it inside and 
outside by a predetermined amount with reference to the 
extracted contour portion. The structure of this 
embodiment is the same as the embodiment of FIG. 2 3 
except that the operation input section 113 can 
designate an amplitude A and a time period T of the 
wobbling. It is of course possibte^ to automatically 
designate them. The system in this embodiment has a 
function of, if given an amplitude A and a time period 



T of wobbling, moving the picture in the contour region 
(or extracted contour lines) at the amplitude A and at 
the time period T and generating a picture to be 
displayed over the original picture. 

The system in this embodiment is also provided 
with the operation input section 113- If the operation 
input section 113 designates an amplitude A and a time 
period T of wobbling, information about the amplitude A 
and time period T is supplied to the display region 
determination section 103. 

The display region determination section 103, if 
given the amplitude A and time period T from the 
operation input section 113, converts the coordinate of 
(xi, yi) of representative point on each contour into a 
coordinate (xi', yi ' ) expressed by the following 
formula (9) and formula (10): 

xi' = xO + A (xi - xO) sin (27i:t/T) (9) 
yi' = yO + A (yi - yO) sin (27Ct/T) (10) 
Here, k is the ratio of the circumference of a 
circle to its diameter, t is time and (xi, yi) is the 
center of gravity of the contour as described in the 
embodiment of FIG. 23. 

According to the system in this embodiment having 
the above structure, a contour region (or all contour 
lines) is obtained by the same procedures as in FIG. 18, 
a display picture obtained by superimposing the contour 
region over the target region of the original picture 
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is generated by the display picture generation section 
104 and the obtained picture is displayed on the 
display unit such as a display by the picture display 
section 105 . 

5 Meanwhile, according to this system, if the 

operation input section 113 designates the amplitude A 
and time period T of wobbling, the information is given 
Q to the display region determination section 103. The 

.{1 display region determination section 103 converts a 

*^ 

,z2 10 coordinate (xi, yi) of the representative point of the 

contour into a coordinate (xi', yi ' ) expressed by the 
formulas (9) and (10). As a result, each representa- 
y tive point (xi, yi) of the contour changes at the 

O amplitude A and time period T of the wobbling. The 

□ 15 display region determination section 103 composes a 

contour region reflecting this change on the original 
picture. The contour region of the picture displayed 
by the display unit changes at the amplitude A and the 
time period T. 

20 Consequently, the comparison and contrast of the 

picture in the contour region and the original picture 
can be easily made, thus facilitating comparing and 
determining the shape at high accuracy. 

The method of realizing the present invention by 

25 means of software using a calcuLat-cr will be roughly 

mentioned. In the software processing, coordinates of 
representative points on the contours are calculated in 



• 
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accordance with formulas (11) and (12) every time the 
time period t passes by a predetermined interval At. 
From the calculated coordinates, a contour region is 
generated by the approximation of a line or curve. 
5 After the original picture data is re-drawn on the 

display, the obtained contour region is re-drawn. By 
making such a program and executing the program using 
□ the calculator, the present invention can be realized. 

'^=1 In this embodiment, it is possible to automati- 

iifl ^0 cally generate a contour region unlike the embodiment 

i s 

i.p FIG. 23. Since the displayed contour region is moved 

r/ periodically, both the vicinity of the contour within 

the picture and the extracted contour region can be 
observed while dispensing with user's trouble. It is 
15 more effective if a transparent contour region is 

provided by an after-image effect by reducing the time 
period T of wobbling. 

In the embodiment of FIG. 31, a picture processing 
apparatus comprises a picture input section 101, a 
contour extraction section 102, a display picture 
generation section 104 and a picture display section 
105 . 

The picture input section 101 and the contour 
extraction section 102 are the same as those shown in 
the embodiment of FIG. 18. According to the system of 
this embodiment, the display picture generation section 
104 replaces a pixel value of a portion of the original 
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picture corresponding to the contour region extracted 
by the contour extraction section 102, with a predeter- 
mined pixel value. 

According to the system having such a structure, a 
5 contour region (or contour lines) is obtained in 

accordance with the same procedures as those in the 
embodiment of FIG. 18. The display picture generation 
□ section 104 replaces a portion of the original picture 

'"-4 corresponding to the contour region extracted by the 

contour extraction section 102, with a. predetermined 
pixel value and thereby generates a display picture. 

Meanwhile, the picture display section 105 
controls the original picture and the picture generated 
by the display picture generation section 104 such that 
15 they are alternately displayed at predetermined time 

intervals. By so doing, the picture generated by the 
display picture generation section 104 and the original 
picture are alternately displayed on the display unit 
by the picture display section 105. 
20 In this way, according to this system, the display 

picture generation section 104 generates a picture 
obtained by replacing a portion of the original picture 
corresponding to the contour region, with a predeter- 
mined pixel value, and the original picture as well as 
25 the picture generated by the display picture generation 

section 104 are alternately displayed on the display 
unit . 
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Due to this, there exists a time period in which a 
contour region is not drawn. As a result, the original 
picture information can be grasped more precisely and 
both the vicinity of the contour within the picture and 
the extracted contour region can be observed while 
dispensing with the user's trouble. Thus, operability 
is improved. Besides, if the predetermined time 
interval t is reduced, then a transparent contour 
region can be provided by an after-image effect and 
effective observation can be advantageously realized. 

The display switching in this embodiment can be 
realized by software processing using a calculator. 
The method will be now described. 

The display switching by means of software 
processing is conducted as follows. If a predetermined 
time interval is At and time t is 2nAt (where n is an 
integer), original picture data is re-drawn on the 
display. On the other hand, if time t is (2n + l)At, 
original picture data is re-drawn on the display and 
then the contour region previously obtained is re-drawn. 
By so doing, the picture generated by the display 
picture generation section 104 and the original picture 
can be alternately displayed. 

The method shown in FIG. 31, different from the 
embodiment FIG. 23, can grasp original picture informa- 
tion more precisely since there exists a time period in 
which a contour region is not drawn. Besides, without 



troubling the user, the vicinity of the contour within 
the picture and the extracted contour region can be 
observed. Furthermore, if the predetermined time 
interval t is reduced, a transparent contour region can 
be advantageously provided by an after-image effect. 

In the embodiment of FIG. 31, the picture 
processing apparatus comprises a picture input section 
101, the contour extraction section 102, the display 
picture generation section 104 and the picture display 
section 105. 

The picture input section 101, the contour 
extraction section 102 and the display picture genera- 
tion section 104 are the same as those in the 
embodiment of FIG. 30. 

According to the system of the present invention, 
the picture display section 105 controls the original 
picture and the picture generated by the display 
picture generation section 104 such that they are 
alternately displayed. However, the picture generated 
by the display picture generation section 104 is 
displayed only when a command is manually given at the 
operation input section 113. For that reason, a button 
(display button) is provided at the operation input 
section 113 for the designation operation. 

According to the system, while the display button 
on the operation input section 113 is being pushed, 
only the original picture obtained from the picture 
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input section 101 is displayed. When the display 
button is released, only the generated picture of the 
picture display section 105 is displayed. Conversely, 
it is possible to display the generated picture of the 
picture display section 105 while the button is being 
pushed and the original picture is displayed when the 
button is released. It is also possible to change the 
display picture from "the generated picture of the 
picture display section 105" to "the original picture" 
to "the generated picture of the picture display 
section 105" to "the original picture" in this order 
every time the button is pushed. Further, operation 
means other than the button such as a slide bar, a dial, 
a mouse and the like may be used. 

In this embodiment, different from the embodiment 
of FIG. 30, the timing for switching between the 
display and non-display of the contour region is freely 
determined. Since the switching can be thus freely 
made, it is possible to display a picture more conform- 
able to the user's demand. 

The sixth embodiment according to the present 
invention will be described. 

The sixth embodiment is the modification of the 
embodiment of FIG. 30. The picture input section 101, 
the contour extraction section 102 and the picture 
display section 105 are the same as in FIG. 18. The 
display picture generation section 104 generates 



a picture by converting the pixel value of a portion of 
the original picture corresponding to the contour 
region extracted by the contour extraction section 102 
in accordance with the following conversion processing. 
Specifically, if the brightness of the original picture 
is (I, I, I) in RGB color representation and the new 
brightness is (I, I, 0) in RGB color representation, 
the conversion is conducted as follows: 

R = I 

G = I 

B = 0 

That is, the original picture is a color picture, and 
an R (red) component, a G (green) component and a B 
(blue) component of the original picture has the same 
brightness value which is I and colorless (either white 
or gray depending on the value of I). By the conver- 
sion processing, the R component and the G component 
are converted into "I" and the B component is con- 
verted into "0", As a result of the conversion 
processing, the monochrome pixel having the R component, 
G component and B component which are all I is con- 
verted into that having only the R component and the G 
component which are yellow. Therefore, after the above 
processing, the color of the contour region is changed 
to yellow. The brightness of the contour region is 
proportional to that of the original picture (that is, 
the brightness of the contour region having "I" is 
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proportional to that of the original picture) . 
Therefore, it is possible to obtain the original 
picture information even in the contour region. In 
addition, since information about the contour extrac- 
5 tion result can be also obtained due to a change in 

color, it is possible to easily evaluate the contour 
extraction result . 

In this embodiment, yellow is used as a color for 
representing the contour region. However, other colors 
,4! 10 may be used. As long as the brightness within the 



contour region can be kept, other conversion methods 
may be used. In this embodiment the B component is 
fixed to 0 (B = 0). However, it may be changed 
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^-3 between "0" and "I" as time passes. In that case, the 

Q 15 change can be obtained by wobbling both periodically 

and manually. If so, the change can give a visual 
impression more strongly, thereby facilitating the 
recognition of the contour region. 

As described so far, the present invention 
20 extracts the contour of a target within a picture, 

divides the extracted contour into a display portion 
and a non-display portion and displays the target 
picture and the display portion within the contour by 
superimposing the display portion over the target 
25 picture. In addition, the present invention extracts 

the contour of the target within the picture, displays 
the extracted contour region by superimposing the 
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contour region over the target picture within the 
picture and displays the contour while scaling it. 
Furthermore, the present invention changes the color of 
the contour when displaying it and can switch the 
5 display picture between the picture, over which the 

contour is superimposed, and the original picture. 
Therefore, the contour picture and the original picture 
can be easily compared, thereby facilitating the 
comparison and determination of the shape at accuracy. 

10 As a result, it is possible to apply the present 

invention to the evaluation of the accuracy of the 
contour extraction result and the analysis of the state 
of the movement of a moving target using the contour 
lines of the picture. 

15 The method described in the embodiment with 

reference to the drawings can be realized as software. 
The program by using the software can be stored in a 
storage medium, which can be read by a computer, such 
as a magnetic disc (a floppy disc, a hard disc), an 

20 optical disc (a CD-ROM, a DVD) and a semiconductor 

memory and can be distributed widely. 

According to the embodiments of the present 
invention described above, it is possible to provide a 
picture processing apparatus and a picture processing 

25 method having an excellent advanta^^j-e in that the 

determination of whether or not the extracted contour 
is correct can be easily made, which the conventional 
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apparatus and method cannot make, and in that it is 
possible to apply the present apparatus and method to 
the analysis of the state of the movement of a moving 
target by using the contour of the picture of the 
5 target since the correct contour can be obtained. 

Now, description will be given to various 
embodiments of a picture processing apparatus and a 
picture processing method capable of automatically 
obtaining, for example, a terminal diastole area or 

M 10 volume and a terminal systole area or volume of an 

"P 

;3 ultrasonic picture. 

M FIG. 33 shows the structure of the ultrasonic 

□ picture processing apparatus. FIG. 34 shows the 

processing flow of a method for detecting picture 
15 collecting conditions of an ultrasonic picture. 

A moving picture input section 201 inputs the 
moving picture of the heart of a subject by means of 
ultrasonic waves, X rays, magnetic resonance or the 
like. A target contour extraction section 202 extracts 
20 the contour of the heart from the heart moving picture 

for every picture (every frame). A contour internal 
area calculation section 203 calculates the area inside 
the extracted contour of the heart. A. maximum/minimum 
area calculation/storage section 204 detects the 
25 maximum/minim areas among the calculated internal areas 

of the contour of the heart and stores the detected 
values while associating them with the pictures 
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corresponding to the values, respectively. 

According to the apparatus in this embodiment 
having this structure, moving pictures of the heart of 
the subject are inputted by the moving picture input 
5 section 201. This is done by obtaining moving pictures 

of the heart of the subject obtained by, for example, 
the ultrasonic diagnosis apparatus and using the 
obtained pictures as inputs. 

More specifically, the moving picture input 

10 section 201 inputs moving pictures of the heart 

obtained in a time-series manner from, for example, the 
ultrasonic diagnosis apparatus (S201). The target 
contour extraction section 202 extracts the contour of 
the target (cardiac wall) from each of the inputted 

15 moving pictures (S202). The extraction of the contour 

of the cardiac wall can be conducted automatically and 
easily by means of, for example, the method using 
active contour models and balloons: "On Active Contour 
Models and Balloons (Laurent D. Cohen, CVGIP: lU, 

20 53(2): 211-218, 1991). FIG. 34 shows the typical 

example of an extracted contour by that method. By 
conducting processing in step S202, contour information 
can be obtained for every picture as shown in FIG. 34. 

Thereafter, processing is conducted by the contour 

25 internal area calculating section l203. The contour 

internal area calculation section 203 calculates the 
internal area of each of the contours based on the 



respective contour information extracted by the target 
contour extraction section 202 (S203). The maximum/ 
minimum area calculation/storage section 204 detects 
and stores maximum/minimum values of the internal area 
of the contour of each picture obtained by the contour 
internal area calculation section 203 (S204). 

Contours are extracted from the time-series 
ultrasonic pictures showing the sectional views of the 
heart and internal areas of the contours are calculated 
using the extracted contour information, thereby 
obtaining the maximum and minimum areas . 

The detailed method for obtaining the maximum and 
minimum internal areas of the contours will be now 
described . 

FIG. 35 shows the example of the processing flow 
of the step S2 04 conducted by the maximum/minimum value 
detection/storage section 204 shown in FIG. 33 in more 
detail. In the step S204, the calculated value of the 
internal area of the contour obtained in step S203 is 
inputted (S208). It is determined whether it is a 
value is for the first picture (S209). If it is 
determined as first picture value, the inputted area is 
temporarily stored in maximum/minimum memories (S210). 

Next, the stored value is compared with a value (a 
retained maximum value) stored in the maximum memory 
(5211) . If the value temporarily stored in the 
maximum/minimum memories S210 is larger than the 



retained maximum value (S212), the maximum value is 
updated to the temporarily stored value in the maximum 
memory (S213 ) . 

Meanwhile, as a result of the determination in 
step S212, if the value temporarily stored in the 
maximum/minimum memories is not larger than the 
retained maximum value, the maximum value is not 
updated and the temporarily stored value is compared 
with a value (a retained minimum value) stored in the 
minimum memory (S214). If the value temporarily stored 
in the maximum/minimum memories in step S210 is 
smaller than the retained minimum value (S215), the 
minimum value in the minimum memory is updated to the 
temporarily stored value (S216). As a result of the 
determination in step S215, if the temporarily stored 
value is not smaller than the retained minimum value, 
the minimum value is not updated and the processing 
enters step S217. 

In the step S217, it is determined whether or not 
the processing of input pictures for a predetermined 
time period is finished. If the processing for the 
predetermined time period is finished, a maximum memory 
value and a minimum memory value are stored as a 
terminal diastole area and a terminal systole area, 
while associating them with each -other, respectively 
(S218). 

The data can be stored in the memory by arranging 



in a form of (terminal diastole area value, the time 
phase thereof), (terminal systole area value, the time 
phase thereof) and the like. It is also possible to 
simultaneously store the electrocardiogram information 
about the corresponding time phases - These arranged 
values may be outputted to an output unit such as a 
printer instead of storing them in the memory or while 
doing so. 

In this embodiment, contours of the heart are 
extracted based on the moving pictures of the real-time 
ultrasonic sectional image of the heart for every 
picture plane. The internal area of each of the 
contours is obtained and it is determined whether the 
area is the maximum or minimum value. The maximum and 
minimum value are updated and retained for a predeter- 
mined time period. Due to this, it is possible to 
detect the maximum and minimum sectional areas of the 
heart shown on the inputted pictures for a predeter- 
mined time period. If the predetermined time period is 
in line with the moving period of the heart, the 
terminal systole and the terminal diastole of the heart 
can be grasped automatically. 

For the purpose of discovering the terminal heart 
systole period and the terminal heart diastole period, 
the following area measurement method is taken. 

FIG. 36 shows an embodiment of a picture process- 
ing apparatus. FIG. 37 shows the processing flow of 



the picture processing apparatus. 

In FIG. 36, the picture processing apparatus 
comprises a contour internal point selection section 

221, a measurement straight line group setting section 

222, a pixel number measurement section 223 and an 
addition section 224 . 

The contour internal point selection section 221 
selects a required point inside the detected contour E. 
The measurement straight line group setting section 222 
sets a measured group of straight lines provided 
radially around the required point. The pixel number 
measurement section 223 measures the number of passed 
pixels p for every straight line by the straight line 
set by the measurement straight line group setting 
section 222 crosses the contour. The addition section 
224 obtains the cross-sectional area of the heart by 
adding the number of pixels p measured by the pixel 
number measurement section 223. 

In such a structure, a point C is selected inside 
the detected contour E (S221). This is conducted by 
the contour internal point selection section 221. The 
internal point C of the contour E may be, for example, 
a point of the center of gravity having a coordinate (x, 
y) represented by the averages of all X and Y coordi- 
nates on the contour E, respectively. 

When the point C is selected, the measurement 
straight line group setting section 222 sets a measured 
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group of straight lines arranged radially around the 
point of the center of gravity, that is, point C (S222) 
The pixel number measurement section 22 3 measures the 
number of passed pixels e for every straight line by 
5 the straight line crosses the contour (S223). At this 

time, the measured pixels are stored so as not to 
measure them a plurality of times. The addition 
section 224 adds the number of pixels e measured by the 
SJ pixel number measurement section 22 3 and obtains the 

iTi 10 cross-sectional area of the heart. 

ijj FIG. 38 typically shows how the processing is 

'''^ 

^/ going on. The contour E of the heart has an almost 

convex, smooth shape. Therefore, if to-be-measured 
straight lines are set densely enough, the cross- 
15 section area of the heart can be accurately calculated 

by the method of this embodiment. 

Another method for obtaining the cross-sectional 
area of the heart will be described with reference to 
FIGS. 39 and 40. A picture processing apparatus shown 
20 in FIG. 39 comprises a contour division section 231, an 

area calculation section 232 and an addition section 
233. The contour division section 231 measures the 
length of a given contour E, divides the length at 
predetermined intervals and obtains each division point 
25 represented as Pt . The area calculation section 232 

uses the internal point C of the contour E obtained by 
the contour internal point selection section 221 in the 



embodiment of FIG. 37, creates a triangle by connecting 
the point C, a representative point Pt and its adjacent 
representative points Pt for every point Pt and 
calculates the area of each triangle. The point C may 
be, for example, the point of the center of gravity 
having a coordinate (x, y) represented by averages of 
all of the X and Y coordinates on the contour E, 
The addition section 233 adds areas of the 
respective triangles obtained by the area calculation 
section 232 and thereby calculates the internal area of 
the contour. 

In the apparatus of this embodiment having such a 
structure, the contour division section 231 measures 
the length of the contour E extracted by the contour 
extraction section, divides the length at predetermined 
intervals and set respective division points Pt as 
representative points (S231). After obtaining the 
respective points Pt, processing enters an area 
calculation step. This step is conducted by the area 
calculation section 232. 

The area calculation section 232 creates a 
triangle by connecting the point CO which is the point 
of the center of gravity, a representative point Pt and 
its adjacent representative point Pt for every point Pt 
(refer to FIG. 41) and calculates - the area of the 
triangle (S232). The point CO of the center of gravity 
has a coordinate (x, y) obtained by averaging all of 



the X and Y coordinate points on the contour, respec- 
tively . 

Finally, the addition section 233- adds areas of 
all of the triangles and thereby calculates the 
internal area of the contour (S233). 

As can be seen from the above, the area can be 
calculated by means of the polygon approximation. 

The above-described methods are intended for 
discovering the heart terminal systole period and the 
heart terminal diastole period by obtaining the area of 
the cross-sectional heart image. However, it is 
possible to grasp the heart terminal systole period and 
the heart terminal diastole period by obtaining not 
only the area but also the volume as follows. 

The description will now be given to methods for 
grasping the heart terminal systole period and the 
heart terminal diastole period by using the volume of 
the heart . 

An embodiment for obtaining the volume of the 
heart from the cross-sectional heart image will be 
described with reference to FIGS. 42 through 44. This 
embodiment adopts a method for obtaining the heart 
terminal systole period and the heart terminal diastole 
period from the volume of the heart. 

As shown in FIG. 42, a picture processing 
apparatus comprises a picture input section 241, a 
target contour extraction section 242, a contour 



internal volume calculation section 243 and a maximum/ 
minimum volume detection/storage section 244. The 
picture input section 241 inputs moving pictures of the 
heart of a subject- The target contour extraction 
section 242 extracts contours of the target heart from 
the moving pictures (frames) of the heart, respectively. 

The contour internal volume calculation section 
243 calculates internal volumes of the contours of the 
heart from the extracted contours of the heart. The 
maximum/minimum volume detection/storage section 244 
detects a maximum volume and a minimum volume of the 
internal volumes of the contour obtained for respective 
pictures (respective frames) and stores the values 
while associating them with the corresponding pictures. 

According to the method for obtaining the heart 
terminal systole period and the heart terminal diastole 
period from the volumes of the heart adopted in the 
embodiment of FIG. 42, a moving picture of the heart is 
first inputted from the ultrasonic diagnosis apparatus 
(S241). The target contour extraction section 242 then 
extracts the contour E of the target (cardiac wall) 
from the inputted moving picture (S242). The contour 
is extracted for every picture (every frame) . 

When the contour for every picture (every frame) 
is extracted, the contour internal volume calculation 
section 242 calculates the internal volume of the 
contour for the extracted contour (S243). The volume 



can be calculated by, for example, obtaining the 
central axis of the contour E, dividing the central 
axis into a plurality of segments each having a 
predetermined distance d, obtaining the volume of a 
circular cylinder having a diameter ri taken from one 
intersection between a segment passing one of the 
division points and perpendicular to the central axis 
and the contour E to another intersection therebetween 
and obtaining the sum of the obtained volumes of the 
cylinders . 

Finally, the maximum/minimum volume detection/ 
storage section 244 detects a maximum volume and a 
minimum volume in the same manner as shown in FIG. 4 3 
and stores the values while associating them with the 
corresponding pictures (frames), respectively. 

By using the volume measurement method, it is 
possible to grasp the heart terminal systole period and 
the heart terminal diastole period automatically. 

There is a method for calculating the movement 
amounts between contours and obtaining a minimum value 
from the calculated movement values while taking it 
into consideration that the contour movement amount is 
a minimum in the terminal systole period and the 
terminal diastole period, and for specifying as a 
terminal diastole area/volume, the larger area/volume 
in the time phases and as a terminal systole area/ 
volume, the smaller area/volume in the time phases. 



An embodiment using this method will be described 
hereinafter , 

As shown in FIG. 34, a picture processing 
apparatus in this embodiment comprises a moving picture 
input section 251, a target contour extraction section 

252, a contour internal area/volume calculation section 

253, a contour movement amount calculation section 254, 
a minimum movement amount detection section 2 55 and a 
storage 256 . 

The moving picture input section 251 inputs moving 
pictures of the heart of a subject. The target contour 
extraction section 252 extracts the contour of the 
heart from every inputted moving picture (every frame) 
of the heart. 

The contour moving amount calculation section 254 

calculates the moving amount of the contours extracted 

by the target contour extraction section 252. The 

minimum movement amount detection section 255 obtains a 

minimum value and maximum value from the contour 

movement amounts obtained by the contour movement 

amount calculation section 254. The calculation 

section 253 calculates internal areas/volumes of the 

contours extracted by the target contour extraction 

section 252, respectively. 

Among those areas /volumes , the larger area/volume 

•A 

and the smaller area/volume are stored as the terminal 
diastole period area/volume and the terminal systole 
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area/volume, respectively while associating them with 
corresponding pictures. That is, a picture showing a 
larger area/volume in a certain time period {, i.e., a 
picture showing a maximum area/volume in a time period 
among the time series pictures (frames) and a picture 
showing a smaller area/volume in a certain time period) 
are stored while associating these pictures with the 
corresponding values, respectively. The storage 256 
stores and retains them. 

In the apparatus having the above structure, 
moving pictures of the heart obtained from the moving 
picture input section 251 by, for example, the ultra- 
sonic diagnosis apparatus are first inputted (S251). 
The target contour extraction section 252 extracts 
contours E of a target (cardiac wall) from the inputted 
time series moving pictures (S252), respectively. 

The calculation section 253 calculates the 
internal areas /volumes of the contours based on the 
extracted contours E, respectively (S253). At the same 
time, the contour movement amount calculation section 
254 calculates contours moving amounts based on the 
extracted contours E (S254). 

Using the fact that the movement amount becomes a 
minimum in the terminal expansion period and the 
terminal systole period, the minimum moving amount 
detection section 255 obtains a minimum value from the 
moving amounts calculated by the contour moving amount 



calculation section 254 (S255). The larger area/volume 
and the smaller area/volume in the above time phases 
are defined as a terminal diastole area/volume and a 
terminal systole area/volume, respectively and stored 
in the storage 256 while associating the values with 
the corresponding pictures (256). 

As described above, by using that the contour 
moving amount becomes a minimum in the terminal 
diastole period and the terminal systole period, 
contour moving amounts are calculated and a minimum 
value is obtained from the calculated moving amounts. 
The larger area/volume and the smaller area/volume in 
those time phases are specified as a terminal diastole 
area/volume and a terminal systole area/volume, 
respectively. The heart terminal diastole period and 
the heart terminal systole period can be thereby 
discovered automatically . 

Another embodiment for grasping the heart terminal 
diastole period and the heart terminal systole period 
will be described with reference to FIGS. 47 and 48. 

This embodiment adopts a method for discovering a 
terminal systole period and a terminal diastole period 
of the heart from the difference in the internal areas 
of the respective heart contours by time-series 
arranging the contours obtained from the moving 
pictures of the heart, and for simultaneously obtaining 
the average movement amounts . 



The structure of this embodiment comprises a 
contour internal area calculation section 261, a 
difference calculation section 262, a contour length 
measurement section 263 and a division section 264, 

The contour internal area calculation section 
261 calculates internal areas of contours from the 
contours of a target extracted from the target contour 
extraction section 252. The difference calculation 
section 262 arranges pictures in a time series manner 
and sequentially subtraction-processes the contour 
internal areas of the pictures, thereby obtaining time 
differences in areas as a result of the subtraction- 
processing conducted for continuous time periods. The 
contour length measurement section 263 counts the 
number of pixels on the contours and thereby measures 
contour lengths. The division section 264 divides time 
differences of area by the contour lengths and thereby 
obtains an average movement amount. 

To obtain the average movement amount of the heart, 
according to this embodiment, contours E of ultrasonic 
cross-sectional images of the heart inputted as moving 
pictures are extracted and internal areas of the 
contours E are obtained, respectively (S261). The 
contour internal area calculation processing is 
conducted by the contour internal area calculation 
section 261. The pictures are arranged in a time 
series manner and the internal areas of the contours E 



are subtraction-processed in the generation order, to 
thereby obtain time differences of area as a result of 
the subtraction-processing conducted for continuos time 
periods (S252). This processing is conducted by the 
difference calculation section 262. 

Next, the contour length measurement section 263 
measures contour lengths by counting the number of 
pixels on the contours E (S263). Finally, the division 
section 264 obtains an average moving amount by 
dividing time differences of areas by the contour 
lengths ( S264 ) . 

As can be understood from the above description, 
it is possible to discover the terminal systole period 
and the terminal diastole period of the heart by 
arranging heart contours obtained from heart moving 
pictures in a time series manner and calculating 
differences in the internal areas of the heart contours. 
It is also possible to obtain useful diagnosis informa- 
tion including the heart movement amounts by obtaining 
an average heart moving amount. It should be empha- 
sized that they can be realized automatically. 

Now, referring to FIGS. 49 and 50, description 
will be given to an embodiment for obtaining the moving 
amount of the heart by adopting a method for obtaining 
characteristic points on contours, for estimating 
movements of the respective characteristic points and 
thereby for obtaining an average movement amount. 



As shown in FIG. 49, the picture processing 
apparatus in this embodiment comprises a characteristic 
point detection section 271, a moving vector detection 
section 272 and a moving amount calculation section 273 
The characteristic point detection section 271 obtains 
characteristic points on contours E extracted by the 
target contour extraction section 252, respectively. 
The moving vector detection section 272 correlates and 
collates a region in the vicinity of the obtained 
characteristic point and a picture in a time phase 
following the time phase of the former picture and 
detects the movement of the difference in coordinate 
between a point having the largest correlation value 
and an original characteristic point as a moving vector 
The moving amount calculation section 273 obtains an 
average moving amount from moving vectors of respective 
characteristic points by statistic processing. 

The apparatus in this embodiment having such a 
structure extracts contours E of cross-sectional images 
of the heart inputted as moving pictures and obtains 
characteristic points of the contours E, respectively. 

To be more specific, the characteristic point 
detection section 271 obtains characteristic points 
based on the contours E of the cross-sectional images 
of the heart to obtain movement amounts (5271). These 
characteristic points may be selected by, for example, 
obtaining a point having a large curvature change on 



a contour E as a characteristic point. Alternatively, 
a variance of adjacent points on a contour E is 
obtained and a point having a value not less than a 
predetermined value may be selected as a characteristic 
point. Furthermore, by dividing a contour E into a 
plurality of parts at predetermined intervals, a 
characteristic point may be selected. 

Processing by the moving vector detection section 
272 then starts. The moving vector detection section 
272 estimates movements of characteristic points on 
contours E. 

In this embodiment, the evaluation is made as 
follows. A region in the vicinity of a characteristic 
point and a picture in the following time phase are 
correlated and collated and detects a difference in 
coordinate between a point having the largest correla- 
tion value and an original characteristic point as a 
moving vector (S272). 

Processing by the moving amount calculation 
section 273 then starts. The moving amount calculation 
section 273 obtains an average moving amount from 
moving vectors of respective characteristic points by 
statistic processing (S273). In this embodiment, the 
averages of x components and y components of the 
respective moving vectors are obt-a-ined and the magni- 
tudes are calculated, whereby an average moving amount 
is obtained. 
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In short, the present invention is characterized 
in that contours of a target are extracted from moving 
pictures obtained by, for example, an ultrasonic 
diagnosis apparatus, that internal areas of the 
extracted contours are calculated and that maximum and 
minimum areas are detected as a terminal diastole area 
and a terminal systole area, respectively. 

Moreover, the picture processing method according 
to the present invention is characterized in that 
contours of the target are extracted from moving 
pictures obtained by, for example, the ultrasonic 
diagnosis apparatus, that contour internal volumes are 
calculated using the extracted contours, and that the 
maximum and minimum values are obtained and detected as 
a terminal diastole volume and a terminal systole 
volume , respectively . 

In addition, the picture processing method 
according to the present invention is characterized in 
that contours of a target are extracted from moving 
pictures obtained by, for example, the ultrasonic 
diagnosis apparatus, that areas or volumes showing the 
smallest moving amount are obtained from moving amounts 
on a time-by-time basis by using those extracted 
contours and that the maximum area/volume and the 
minimum area/volume out of the obtained areas/volumes 
are detected as a terminal diastole area/volume and a 
terminal systole area/volume, respectively. 



According to the present invention, areas or 
volumes are calculated from cross-sectional heart 
images obtained as moving pictures based on information 
about the extracted heart contours. The calculated 
areas /volumes are compared in a time series manner. As 
a result, it is possible to properly determine the 
terminal diastole period and the terminal systole 
period of the heart. 

According to the present invention, concrete areas 
or volumes of the heart are obtained and, based on 
these obtained values, the terminal diastole period and 
the terminal systole period of the heart can be 
properly determined. This makes it possible to obtain 
objective measurement information useful to the 
diagnosis of the subject's heart function. Besides, 
since the information can be obtained automatically, 
the burden on the operator can be reduced. 

It is noted that the present invention should not 
be limited to the above-described embodiments. Various 
modifications are possible. Furthermore, not only 
ultrasonic pictures but also cross-sectional pictures 
of CT scan or MRI as well as X-ray television pictures 
can be used in the present invention. It may be 
possible to use the present invention in fields other 
than the medical field. 

Since the present invention utilizes moving 
pictures, pictures (or frame pictures) changing moving 



states in accordance with heartbeats can be inputted in 
a time series manner. Heart contours of those pictures 
in respective time phases are extracted and processed . 
However, it is not necessary to use all of the frame 
pictures. Instead, among the time series pictures (or 
frame pictures), some of the pictures which reflect 
moving states in a systole period and an diastole 
period may be selected and processed. Therefore, 
various modifications are possible depending on the 
circumstances . 

The methods described with reference to the 
flowcharts of FIGS. 33, 35, 37, 40, 43, 46, 48 and 50 
can be stored in a storage medium including a magnetic 
disc (such as a floppy disc and a hard disc) and an 
optical disc (such as a CD-ROM and a DVD) as programs 
which can be executed by a computer and distributed 
widely . 

According to the embodiments of the present 
invention, it is possible to automatically obtain 
either areas or volumes or both of them in the terminal 
diastole period and the terminal systole period of the 
heart for the measurement of the pumping function which 
is important to the diagnosis of a disease, by using 
moving pictures obtained by, for example, an ultrasonic 
diagnosis apparatus. Therefore, if cross-sectional 
heart images are inputted as moving pictures, for 
example, then the terminal diastole period and the 
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terminal systole period of the heart can be accurately 
determined from concrete values of the obtained areas 
and/or volumes based on the cross-sectional heart 
images, and the heart pumping function can be quantita- 
tively obtained. Thus, the present invention can 
advantageously facilitate using objective measurement 
information which is useful to the diagnosis of the 
subject's heart function. 

Now, description will be given to a heart function 
analysis support apparatus and method for accurately 
associating the cardiac wall contours and facilitating 
the evaluation of local movement states of the cardiac 
wall so that the display of information calculated from 
the contour information which can be easily evaluated 
is realized. 

A heart function analysis apparatus shown in 
FIG. 51 comprises a cardiac wall contour input section 
301 for inputting cardiac wall contour information, a 
characteristic point detection section 302 for detect- 
ing or inputting characteristic points on contours such 
^ cardiac apex and an annulus valva from the hear \^ 
wall contours, a contour division section 303 for ' 
dividing the cardiac wall contours based on the 
characteristic points, a division point association 
section 304 for associating division points of moving 
pictures in a plurality of time phases, a display 
section 305 for classifying the divided cardiac wall 
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contours into^egions useful for diagnosis and for 
displaying the divided contours by means of at least 
one of numerical display, graph display, color display 
of the cardiac wall and a memory for storing contour 

5 information or division point information. 

In the present apparatus having the above 
structure, the cardiac wall contour input section 301 
inputs cardiac wall contour information. Time ser ies 
cross -sectional imag es of the subject's heart are 

0 obtained from, for example, the ultrasonic dia gn osis 

aggar atus . Contours of the heart are extracted based 
on the obtained images - The extracted contours are 
used as the cardiac wall contour information. The 
cardiac wall contour information is stored in the 

5 memory 6. The cardiac wall contour information is fed 

to the characteristic point detection section 302. 

When the cardiac wall contour information is 
inputted, the characteristic point detection section 
302 detects characteristic points on contours such as a 

0 cardiac ape x and an annulus valva from the cardiac wall 

contours based on the cardiac wall contour information. 
This can be automcr^ calcu lated by using the 

cur vat ure of^con tours based on the sha p es of th^ _ 
cardiac wall contours (automatic detection processing). 

5 It is also possible to manually input characteristic 

points using, for example, a mouse (manual input 
operation) , 



The contour division section 303 divides the 
cardiac wall contours based on the characteristic 
points^ obtained by the characteristic point detection 
section 302. The division information obtained by the 
division processing is stored in the memory 6. When 
the cardiac wall contours are divided, the division 
point association section 304 associates the division 
poj.nts on the contours of pictures in a plurality of 
time phases. The display section 305 classifies the 
divided cardiac w all conto urs into regions useful for 
diagnosj ^^jid dj.splav£_them by means of at_least one of. 
numer ic a 1 d is pi ay^,^ graph display and color display of 
the cardiac wall. 

The detailed processing of the heart function 
analysis apparatus having the above structure will be 
described with reference to the flowchart of FIG. 52. 

First, cardiac wall contour information is 
inputted from the cardiac wall contour input section 
301. The cardiac wall contours may be inputted 
manually using, for example, a mouse on pictures or may 
be contour information obtained as a result of picture 
processing and the like (FIG. 53 and step Al of 
FIG. 52). 

Next, characteristic points on the cardiac wall 
contours are detected by the charorcteristic point 
detection section 302. The characteristic point may be 
inputted manually using, for example, a mouse or 



calculated automatically using information about the 
c ur va t u r e o f _c_on t ou r based on the shapes of the 
cardiac wall contours (FIG. 54 and step A2 of FIG. 52). 

The inputted cardiac wall contours are then 
divided based on the detected characteristic points 
(processing conducted in the contour division section 
303) . 

The division of the cardiac wall contours is 
conducted by the following method. 

This embodiment illustrates a method for dividing 
cardiac wall contours while a point of a cardiac apex 
and an annulus valva ^^30_2^ are used as characteristic 
points... 

As shown in FIG. 55, a portion from a right 
^ annulus valva to a cardiac apex is divided into n parts 
and a portion from the cardiac apex to a right annulus 
\ valva is divided into m„ parts . Division points 322 are 
allotted with particular numbers, respectively and 
stored (Step A3 of FIG. 52). The division numbers m 
and n can be appropriately set in accord ance with the 
density ^£or analyzing the movement of the cardiac wall. 

The above-described processing steps are conducted 
for contour data in all time phas es (Step A4 of 
FIG. 52). The division of the cardiac wall contours is 
finished . 

When the cardiac wall contours are divided, 
division points for respective time phases are 



associated. More specif icall y, divis ion points having 
tjie^ same number are associated with one^ another (Step^ 
A5 of FIG. 52). This step is conducted by the division 
point association section 304. 

As described above, according to the embodiment 
shown in FIGS. 51 and 52, the cardiac apex and the 
annulus valva of the hea rt having shapes of definite,^, 
chara(^ers_^j;e_u^s points. Due to 

this, positions of cardiac apex and the annulus valva 
can be accurately associated. Furthermore, according 
to the embodiments thereof, the cardiac wall contours 
are divided based on the cardiac apex and the annulus 
valva. Due to this, as shown in FIG. 56, association 
of the wa ll contours in two time periods can be made 
more appropriately than that by the center line method . 

The results of the association of division points, 
if finished, are displayed (Step A6 of FIG. 52). In 
this case, as shown in FIG. 57, the cardiac wall 325 is 
displayed. Inside the cardiac wall 325, division 
points^from left a n n u 1 u s va 1 v a t o the cardiac apex and 
those from the cardiac apex to the right annulus valva 
are further classified into three parts, respectively. 
Cont^u£^lj.njs wall contours 32 0) are divided 

by colors A, B, C, D, E and F for respective classified 
parts and superimposed over the cross-sectional heart 
pictures . 

The three classified parts may be, for example. 



a base portion, a central portion and a cardiac apex 
portion so as to be useful for diagnosis. By classify- 
ing the^^ardiac wall and displaying the divided contour 
lines by coloring them for respective classified parts, 
it is possible to appropriately divide the cardiac wall 
region and to easily and definitely discover the 
position of the cardiac wall region on the picture. 

There is a different display method. A contour El 
in a certain time phase (a certain time period) as a 
Reference time phase is pre-set with respect to the_ 
periodic diastole and systole movement of jthe heart . 
As shown in FIG. 58, division points on a contour El in 
the reference time period and those on a contour E2 
(which is one of presen t ta rget contours and a contour 
in a di fferent time phase) in a present time phase are 
connected to one another by straight lirves_^, respec- 
tively. By so doing, it is possible to easily grasp 
which part of the cardiac wall region moves to what 
degree with respect to the cardiac wall contour in the 
preference time phase. 

It is also possible to display contour division 
results of pictures in a plura .lity of continuous- time 
phases as moving pictures . By using the moving picture 
dis play the movement state of the cardiac 

wall can be displayed such that they can be grasped 
more easily. FIGS. 57 and 58 show an example of 
displaying the contours by classifying divided parts 
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with different five colors of A to F. However, this is 
nothing but one example. It is not always necessary to 
display all contours with different colors. As long as 
the division parts of the contours can be recognized, 
it is not required to color the contours. Different 
types of lines may be used for classification . 
Alternatively, such a display method as to display at 
least adjacent contours wijth different colors so .as .t.p.^ 
^discriminate contours, may be adopted. 

In the meantime, since a heart is ellipsoidally 
sphere, there are a cross section along a longer axis 
and a cross section along a shorter ^axis , The above 
descriptions have been described while using a cross 
section along a lojiger_ axis as a cross-sectional heart 
image . The same processing is possible even if using a 
cross section along a shorter axis__as a cross-sectional 
heart image as shown in FIG. 59. In that case, 
accurate association can be realized by using, as a 
characteristic point, a papillary muscle which is one 
tissue of the heart. By using this, it is possible to 
even analyze movements which are not perpendicular, to. a 
cardiac wall contour such as a heart torsion movement. 

In the embodiment shown in FIGS. 51 and 52, the 
cardiac^ v^lJ^conltours^ are divided into a predetermined ^ 
number of pajrts to set division pcr±-nts and the cardiac 
wall CO n t o u r s J.n tijne_peri^ds_ of the beating heart are 
displa yed for respec^tive division points to clarify the 
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division parts. In the embodiment, as characteristic 
points serving as reference points to the division 
parts on the heart contours, the cardiac apex and the 
annulus valva of the heart having shapes of definite 
5 characters are used. 

Next, description will be given to an embodiment 
wherein division points are set by dividing cardiac 
p jvall contours into a predetermined number of parts, 

vj \ ^movement distances of the division points in respective 

fp. 10 \ time periods of the beating heart from the correspond- 

ing division points in a reference time per.iod are 
calculated and the information about movement distances 
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of respective division points is displayed whil 
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reflecting on the pictures. In this case, cardiac apex 
15 and the annulus valva of the heart having shapes of 

definite characters are used as characteristic points 
serving as reference points to the division points on 
the heart contours . 

As shown in FIG. 60, the apparatus in this 
embodiment comprises a cardiac wall contour input 
section 301 for inputting cardiac wall contour informa- 
tion, a characteristic point detection section 302 for 
detecting or inputting characteristic points on cardiac 
wall contours such as the cardiac apex and the annulus 
25 valva, a contour division section 303 for dividing the 

cardiac wall contours based on the characteristic 
points, division point association section 304 for 
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associating division points of the contours from 
pictures in a plurality of time phases, a display 
section 305 for classifying the divided cardiac wall 
cont ours in to parts useful for diagnosis and display 
5 them by means of at least one of numerical display, 

graph display and color display of the cardiac wall, a 
memory 306 for storing contour information or division 
point information and a movement distance calculation 

•p^ section 307 for calculating movement distances of 

^^-1 10 respective division points. 

£ s s 

Q That is, the structure of the apparatus in this 

embodiment is characterized by comprising the^moveme nt 
□ distance calculation section 307 for calculating 

;33.a -u.^ - _ ^ . _ _ ' ■ ■ - — I I — -I — ■ „ 

! "rr 

Q movement distances of respective division poin ts in 

n 15 addition to^ the structure of tjie embodiment FIG . 51. A 

new display function of reflecting movement distances 
of the division positions of the divided cardiac wall 
contours is also added to the display section 305. 

In the apparatus according to this embodiment, the 
20 inputted cardiac wall contours in respective time 

phases, that is, cardiac wall contours of respective 
time series pictures are divided based on such charac- 
teristic points as the cardiac apex and the annulus 
valva and the pictures in a plurality of time phases, 
25 that is, time series pictures are=associated with one 

another. These procedures are the same as described in 
the embodiment of FIG. 51 (in steps Bl to B5 of 



FIG. 61). 

After dividing the cardiac wall contours, movement 
distances of division points in respective time phases 
from the corresponding division points in the pre-set 
time period are calculated, respectively (Step B6 of 
FIG. 61). The calculation step is conducted by the 
movement distance calculation section 307. Movement 
distances of the division points in respective time 
phases from the corresponding division points in the 
reference time period will be obtained as follows. 

Among time phases of the beating heart, an i-th 
time phase is highlighted. A coordinate of the n-th 
division point of the cardiac wall contour on the 
cross-section of the heart in the i-th time phase on 
the picture is represented as 

(X, Y) = (Xi,n, Yi,n) (11) 
If a reference time phase is the o-th time phase and a 
time phase for calculating a movement distance is the 
j-th time phase, the movement distance d j , n of the n- 
th division point is calculated as 

^j,n = ^j{^j,n - Xo,n)2 + (Yj^n " ^o,n)^ (12) 

Based on the calculation formulas, movement 
distances are calculated. Next, in accordance with the 
calculated movement distances, the display section 305 
gives the cardiac walls contours colors, superimposes 
the colored contours over the cross-sectional heart 
picture and displays them (Step B7 of FIG. 14). In 
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regard of coloring, if the color phase on the n-th 
division point in the j time phase is defined as Cj,n 
as in the case of the formula (3), the portion of the 
cardiac wall having a small movement distance is 
5 colored blue and that having a large movement distance 

is colored red, for example. By so doing, it is 
possible to grasp the movement state of the cardiac 
wall visually, which greatly helps understand the 
,n movement state of the cardiac wall. 



10 Cj,n C) = Mod (k-dj,n, 360) (13) 

Here, k denotes a constant and Mod (,) denotes a 



^ Ja:\Jv reminder . 



As shown in FIG. 62, movement di stances for 

contours (division points of contours_)__are expre^ssed on 
;:J 15 a graph of FIG. 62). Alternatively, a cardiac wall 

H"^ contour is classified into a plurality of parts and the 

-^_^tai:istic of every classified 

cardiac wall part is calculated and is displayed 

>S v/^ — - ■ ' ■ ' 



m 



numerically (a of FIG. 62) 



20 By thus displaying numerical information, it is 

possible to obtain quantitative information. If, for 
example, an average value for every^part of the heart 
cont ours is^ai^culated^as^ sta^^t^ the average Ej,i 
of the i-th cardiac wall part of the picture in the i- 
25 th time phase is expressed as 

.... . ^ 

= N 2:dj,n (14) 

n€i th cardiac wall 

where N denotes the number of division points belonging 
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to the i-th cardiac wall part. 

As shown in FIG. 63, a change in movement 
distanc es is expressed in a time series manner and a 
time series change may expressed on a graph. Further- 
more, velocities of^the cardiac wall parts can be 
calculated by differentiating movement distances based 
on time phases. A velocity of the cardiac wall part 
perpendicular to an ultrasonic beam cannot be obtained 
as a Doppler signal in the ultrasonic diagnosis 
apparatus. However, by_ adopting the above method, it 
J^^^^^p£ssable to. .obtain velocity information of cardiac 
wall parts irrespectively of the direction of the beams. 

As shown in FIG. 64, a positional difference is 
obtained for every division point between the contour 
15 321 in time phase A and the contour 322 in time phase B 

and the differences are expressed on the picture as 
areas. If movement areas for the respective cardiac 
wall portions are calculated based on division points 
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and expressed, the state of systole movement can be 
grasped for every cardiac wall portion. This is very 
useful as diagnosis information. 

According to the embodiment of FIG. 60, the 
cardiac wall contours are divided into a plurality of 
parts and divided parts are determined. Movement 
25 distances of divided parts in time phases of the 

beating heart, from corresponding divided parts in the 
reference time phase are calculated, respectively. 
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Information about the movement distances of the 
contours for divided parts is reflected on the picture 
and displayed. 

Now, description will be given to an embodiment 
wherein ^oppler information is used, cardiac wall 
contours are divided into a plurality of parts and 

.ipf o^iTi^tion based on the Doppler inf ormation 
for eve ry part is obtained and an picture is displayed 
by reflecting the obtained velocity information for 
every part of the cardiac wall contours on the picture. 
:3 As shown in FIG. 65, the apparatus in this 

embodiment comprises a cardiac wall contour input 
section 301 for inputting cardiac wall contour informa- 
5 tion, a characteristic point detection section 302 for 

15 detecting or inputting characteristic points on 

contours such as a cardiac apex and an annulus valva, a 
contour division section 303 for dividing cardiac wall 
contours based on characteristic points, a division 
point association section 304 for associating division 
points in different time phases with one another, a 
^ ^ display section 305 f or classifying the divided cardiac 
^ . v^ wall contours into a plural ity o_f „p,arts_usef ul for 

^ diagnosis and displayin g th e parts by means of at least 

one of numerical display, graph display and color 



20 



25 display of cardiac wall, a memory ~396 for storing 

contour information or division point information and a 



velocity information input sect ion 308 for inputti 
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velocity information obtained from Doppler signals in 
the ultrasonic diagnosis apparatus. 

To be more specific, the structure of the 
embodiment FIG. 65 is characterized by comprising the 
veloci ty information input section 308 for inputting 
velocity information obtained from Doppler signals in 
^ J^^££f£I} ^ ^ , ji-^S Ilg.^ ^ apparatus in addition to the 
elements of the apparatus in the embodiment of FIG. 51. 
Additionally, the display section 305 is provided with 
\a function ^f c las s i f y ing_veloc ity information about 
t i s s u^^ _o b ta,i n e d from ^Do pp 1 er signals for every cardiac 
^^^^ ^ Pgj^^ usi^g division points of the divided cardiac 
wall contours and displaying the velocity information 
on the division points, respectively. 

The processing by the apparatus in FIG. 65 will be 
described with reference to the flowchart of FIG. 66. 
The apparatus divides cardiac wall contours in respec- 
tive inputted time phases based on characteristic 
points such as a cardiac apex and an annulus valva (in 
steps CI to C3 of FIG. 66). This processing steps are 
the same as those in the first embodiment. In this 
embodiment, information obtained from Doppler signals 
is inputted (Step C4 of FIG. 66). That is, by using 
the ultrasonic diagnosis apparatus having a Doppler 
signal measurement function, Doppler signals of the 
heart are obtained. The velocity information input 
section 308 obtains velocity information from the 



Doppler signals and inputs the velocity information. 

Thereafter, using ^iyig ion points, of the divided 
cardiac wall contours, velocity informat ion .about 
tissues obtained from Doppler signals is classified for 
every cardiac wall part and displayed on resgiective 
division poijits (Step C5 of FIG- 66). The step is 
conducted by the display section 305. 

The detailed displayed contents are as follows. 
By way of example, description will be given to a case 
of classifying a part from a left annulus valva to a 
cardiac apex and a part from the cardiac apex to a 
right annulus valva into three parts, respectively. 

First, a plurality of division points are provided 
on cardiac wall contours. Using the division points, 
the cardiac wall contours are classified into a 

^ required distanced 
For example, a part from the left annulus valva to the 
cardiac apex of a cardiac wall contour and a part from 
the cardiac apex to a right annulus valva are classi- 
fied into three regions; i.e. a base, a central portion, 
a cardiac apex portion in the division order, respec- 
tively. 

Next, velocity information on the respective 
division point s is added _f or _ever v clas sified region 
^j^j^^" ^^Agjjj:^!^^' If-the velocity on the 

n-th division point is Dopplern, the average Ei for 
every cardiac wall part is calculated by formula (5) as 
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follows : 

Mi = i 2]Dopplern (15) 

nei th cardiac wall 

The calculated averages in the respective regions 

are displayed by means of graph display, numerical 

5 display or color display of the cardiac wall portions 

as in the case of the second embodiment (see FIG. 67). 

As described above, the cardiac apex and the 

annulus valva having shapes of definite characters are 

used as characteristic points serving as reference 

□ 10 points for dividing the cardiac wall. Based on these 
'"'•4 characteristic points, the cardiac wall contours are 

□ classified into a plurality of parts and velocity 

p information based on Doppler information is obtained 

Q for every classified part, thereby displaying an 

15 picture reflecting the obtained velocity information 

for every cardiac wall contour part. As a result, 
velocity information about the cardiac wall is dis- 
played for every appropriately classified cardiac wall 
part. Thus, it is possible to display information 
20 useful for the analysis of the cardiac function and to 

greatly contribute to the diagnosis of the cardiac 
function . 

In the meantime, in the display method based on 
velocity information, there are cases where differences 
25 in the velocities of the cardiac wall parts cannot be 

observed very clearly if the differences are small. 
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and analysis is therefore difficult to make. In 
considering this, description will be now given to 

another embodiment . 

^ — 

In this embodiment, cardiac wall contours are 
divided into a plurality of parts, velocity information 
based on Doppler information is obtained for every part, 
^S^XS^I^-^-^^^'^^-^Pf^.the obtained velocity information is 
obtained to thereby change colors within the dynamic 
ran ge . B y s o^ do i n g , if a picture reflecting every 
ca rdia c wall contour part is displayed, colors are 
allotted in accordance with the dynamic range of 
velocities. As a result, even if differences in 
velocities of divided parts are small, they can be 
clearly observed. 

As shown in FIG._j6 8^ the apparatus in this 
embodiment comprises a cardiac wall contour input 
section 301 for inputting cardiac wall contour informa- 
tion, a characteristic point detection section 302 for 
detecting or inputting characteristic points on 
contours such as a cardiac apex and an annulus valva, a 
contour division section 303 for dividing cardiac wall 
contours based on the characteristic points, a division 
point association section 304 for associating division 
points in a plurality of time phases, a display section 
305 for classifying the divided cardiac wall contours 
into a plurality of parts useful for diagnosis and for 
displaying them by means of at least one of numerical 
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display, graph display and color display of the cardiac 
wall, a memory 306 for storing contour information or 
division point information, a velocity information 
input section 308 for inputting velocity information 
5 from Doppler signals in the ultrasonic diagnosis 

apparatus, a dynamic range detection section 309 for 
^o^"^ 1^ detecting a dynam ic rang e and a display color allotme nt 
section 310 for allotting display colors. 

That is, the structure of the embodiment in 
10 FIG. 65 is characterized by comprising the dynamic 

range detection section 309 and the display color 
allotment section 310. 

The processing by the apparatus of FIG. 68 will be 
described with reference to the flowchart of FIG. 69. 
15 In the apparatus of this embodiment, inputted 

cardiac wall contours in respective time phases are 
divided into a plurality of parts based on characteris- 
tic points such as a cardiac apex and an annulus valva 
and the divided parts of pictures in a plurality of 
time phases are associated with one another (in steps 
Dl to D5 of FIG. 69). The processing steps are the 



20 



n 



same as those in the embodiment of FIG. 51. 

^.^^^ ^ ^ After dividing the cardiac wall contours, movement 

y^^^ dxstances of divided parts\ from the correspondinq 

^2 5^ divide.d' parts in the ref erence^^time phase jire caj^cu- 

fJ^ ~'' ' ^ ^ ■- ^..^ - 

^ ^ V " ^^^^^^^^i^J^^specti^/Bly , which processing steps are also 

\ > 'N 

r;- the same as those in the embodiment of FIG. 51] (Step D6 

_ J 
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of FIG.^^) . 

Next, velocity information obtained from Doppler 
signals is inputted (Step D7 of FIG. 68). The input 
step is conducted by, for example, obtaining Doppler 
5 signals of the heart using an ultrasonic diagnosis 

apparatus having a Doppler signal measurement function, 
obtaining velocity information at the velocity informa- 
tion input section 308 and inputting the velocity 

— c^/ ^ 

information. V ^ 

10 Thereafter, the dynamic range detection secti^ori"^ 



309 detects dy namic ranges of movement distances^of 

i?iyA£^°^ points ( Step D8~bf FIG. 69). This 

is done by, for example, calculating differential 

values D'j,n of movement distances of respective 
15 division points in a plurality of time phases or one 

time phase and detecting a maximum value Vmax and a 

minimum value Vmin from the calculated differential 

values D'j,n as follows: 

Vmax = max(D' j,n) (16) 
20 Vmin = min(D' j,n) (17). 

After completing the processing at the dynamic 

range detection section 309, display colors are 

allotted to the Vmin and Vmax (Step D9 of FIG. 69). 

The display colors are allotted by formula (18) by 
25 which a color phase Cj,n of the -display color on the n- 

th division point of the j-th time phase picture is 

defined by a dynamic range, as follows: 
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Cj,nr) = Moci(k-— + Vn,in,360) (18) 

^max ** ^min 

Here, "(°)" in Cj,n (° ) indicates that CJ,n expresses an 
angle . 

The processing as expressed by the formula (18) is 
5 conducted by the display color allotment section 100, 

thereby completing allotting display colors. 

Finally, the display section 305 displays velocity 
information on the picture by using the allotted 
display colors (Step DIO of FIG. 68). 
==3 10 By so doing, even if velocity distribution is 

small, it is possible to display velocity information 
O using many colors. This is useful in diagnosing 

rii 

□ differences in cardiac wall portions in detail. In the 
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display method based on velocity information, there are 
15 cases where differences in movement state cannot be 

observed very clearly if rate differences are small 
among cardiac wall portions, and therefore analysis is 
difficult to make. In the fourth embodiment, cardiac 
wall contours are divided into a plurality of parts and 
20 velocity information based on Doppler information is 

obtained for respective divided parts, the dynamic 
range of the resultant velocity information is obtained 
and colors are changed within the dynamic range. As a 
result, if an picture reflecting the respective parts 
25 of the cardiac wall contours is displayed, it is 

possible to display the picture while allotting colors 



there is only a little difference in velocity, it is 
possible to display the picture while clarifying 
velocity differences among the respective parts. Thus, 
in the analysis of the movement function, the present 
invention can provide an analysis support apparatus and 
an analysis support method capable of grasping the 
movement function clearly. 

The methods shown in FIGS, 52, 61, 66 and 68 can 
be stored as programs executed by a computer in a 
storage medium including a magnetic disc (such as a 
floppy disc and a hard disc) and an optical disc (such 
as a CD-ROM and a DVD) and can be distributed widely. 

According to the present invention described so 
far, cardiac wall parts in various time phases can be 
appropriately associated in the analysis of movement 
state of a cardiac wall by a plurality of time phase 
pictures. Using the division of cardiac wall contours, 
cardiac wall parts are classified by a method suitable 
for a diagnosis, whereby information useful for 
analysis can be displayed. Thus, the present invention 
can provide a cardiac function analysis support 
apparatus and a cardiac function analysis support 
method capable of greatly contributing to the cardiac 
function analysis . 

Additional advantages and mod:if ications will 
readily occur to those skilled in the art. Therefore, 
the invention in its broader aspects is not limited to 
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the specific details and representative embodiments 
shown and described herein. Accordingly, various 
modifications may be made without departing from the 
spirit or scope of the general inventive concept as 
defined by the appended claims and their equivalent. 



